Jul 05 13:18:15.245945 ip-10-0-143-21 systemd[1]: Starting Kubernetes Kubelet... Jul 05 13:18:15.672776 ip-10-0-143-21 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:18:15.672776 ip-10-0-143-21 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 05 13:18:15.672776 ip-10-0-143-21 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:18:15.672776 ip-10-0-143-21 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 05 13:18:15.672776 ip-10-0-143-21 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 13:18:15.675339 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.675258 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 05 13:18:15.682310 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682292 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:15.682310 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682308 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:15.682310 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682313 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682316 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682320 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682322 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682325 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682328 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682331 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682333 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682336 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682338 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682341 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682350 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682352 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682355 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682358 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682361 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682363 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682366 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682368 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682370 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:15.682437 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682373 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682375 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682378 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682380 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682383 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682385 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682388 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682390 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682392 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682395 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682397 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682400 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682403 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682407 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682410 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682413 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682417 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682422 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682424 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682427 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:15.682933 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682430 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682432 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682435 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682438 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682441 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682443 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682446 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682448 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682451 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682453 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682456 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682458 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682460 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682463 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682465 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682467 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682470 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682473 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682475 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:15.683418 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682477 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682481 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682485 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682487 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682490 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682493 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682497 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682502 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682505 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682507 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682510 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682513 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682515 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682518 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682520 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682523 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682525 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682528 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682531 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:15.683932 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682534 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682536 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682539 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682541 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682544 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682546 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682549 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682551 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682553 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682556 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682558 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682560 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682562 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682565 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682567 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682570 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682572 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682574 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.682577 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683334 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:15.684702 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683341 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683345 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683348 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683352 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683354 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683357 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683360 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683364 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683367 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683369 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683371 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683374 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683377 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683380 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683383 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683385 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683388 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683391 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683393 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683396 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:15.685218 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683398 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683401 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683403 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683406 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683408 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683412 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683416 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683418 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683422 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683424 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683427 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683431 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683434 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683437 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683439 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683442 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683444 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683447 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683449 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:15.685742 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683452 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683456 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683460 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683463 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683465 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683468 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683471 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683473 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683475 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683478 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683481 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683483 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683486 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683488 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683490 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683493 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683496 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683498 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683500 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683503 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:15.686215 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683505 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683508 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683510 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683512 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683515 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683519 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683522 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683524 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683527 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683530 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683532 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683534 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683537 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683539 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683542 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683544 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683547 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683549 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683551 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683554 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:15.686720 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683558 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683560 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683563 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683565 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683568 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683570 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683574 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683577 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683580 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683583 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683586 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683588 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683591 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683593 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683595 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683598 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683600 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683604 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.683606 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:15.687219 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684916 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684931 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684935 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684938 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684941 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684943 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684946 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684949 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684953 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684955 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684958 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684960 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684963 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684965 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684968 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684970 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684974 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684978 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684981 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:15.687701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684984 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684986 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684989 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684992 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684995 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.684997 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685000 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685002 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685005 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685008 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685010 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685014 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685017 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685020 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685023 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685026 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685028 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685031 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685033 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685035 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:15.688173 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685038 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685040 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685043 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685045 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685048 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685050 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685052 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685055 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685057 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685060 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685062 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685065 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685067 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685071 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685074 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685076 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685079 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685083 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685087 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685089 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:15.688651 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685092 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685094 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685097 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685099 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685102 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685106 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685108 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685111 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685113 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685116 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685118 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685121 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685124 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685143 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685147 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685150 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685153 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685156 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685159 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685164 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:15.689150 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685167 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685170 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685172 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685175 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685183 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685186 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685188 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685191 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685195 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685199 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685202 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685205 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685208 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685211 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685214 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685216 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685219 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685222 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685225 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685227 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:15.689671 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685301 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685308 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685315 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685320 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685324 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685326 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685331 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685335 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685337 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685340 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685343 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685346 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685349 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685352 2579 flags.go:64] FLAG: --cgroup-root="" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685355 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685357 2579 flags.go:64] FLAG: --client-ca-file="" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685360 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685368 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685373 2579 flags.go:64] FLAG: --cluster-domain="" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685375 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685378 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685381 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685384 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685388 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 05 13:18:15.690210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685391 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685393 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685396 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685399 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685402 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685405 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685407 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685410 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685414 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685417 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685420 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685422 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685425 2579 flags.go:64] FLAG: --enable-server="true" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685427 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685432 2579 flags.go:64] FLAG: --event-burst="100" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685435 2579 flags.go:64] FLAG: --event-qps="50" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685437 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685440 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685443 2579 flags.go:64] FLAG: --eviction-hard="" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685447 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685449 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685452 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685455 2579 flags.go:64] FLAG: --eviction-soft="" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685457 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685460 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 05 13:18:15.690788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685462 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685471 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685474 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685476 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685479 2579 flags.go:64] FLAG: --feature-gates="" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685484 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685488 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685490 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685493 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685496 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685499 2579 flags.go:64] FLAG: --help="false" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685501 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-143-21.ec2.internal" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685504 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685509 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685512 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685515 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685518 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685521 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685524 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685526 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685529 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685532 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685535 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685537 2579 flags.go:64] FLAG: --kube-reserved="" Jul 05 13:18:15.691390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685540 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685543 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685546 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685548 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685551 2579 flags.go:64] FLAG: --lock-file="" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685554 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685556 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685559 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685564 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685567 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685575 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685578 2579 flags.go:64] FLAG: --logging-format="text" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685580 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685585 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685587 2579 flags.go:64] FLAG: --manifest-url="" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685590 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685594 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685597 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685601 2579 flags.go:64] FLAG: --max-pods="110" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685604 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685606 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685610 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685613 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685615 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685618 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 05 13:18:15.692020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685621 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685629 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685632 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685635 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685638 2579 flags.go:64] FLAG: --pod-cidr="" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685640 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685646 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685648 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685651 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685654 2579 flags.go:64] FLAG: --port="10250" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685657 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685659 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c9a6d6a4b79fdc84" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685662 2579 flags.go:64] FLAG: --qos-reserved="" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685665 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685667 2579 flags.go:64] FLAG: --register-node="true" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685670 2579 flags.go:64] FLAG: --register-with-taints="" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685673 2579 flags.go:64] FLAG: --registry-burst="10" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685690 2579 flags.go:64] FLAG: --registry-qps="5" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685701 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685704 2579 flags.go:64] FLAG: --reserved-memory="" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685709 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685712 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685714 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685717 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685720 2579 flags.go:64] FLAG: --runonce="false" Jul 05 13:18:15.692710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685722 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685725 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685728 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685731 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685734 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685737 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685740 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685742 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685745 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685748 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685751 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685754 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685757 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685759 2579 flags.go:64] FLAG: --system-cgroups="" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685762 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685768 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685770 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685773 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685777 2579 flags.go:64] FLAG: --tls-min-version="" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685779 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685782 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685784 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685787 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685790 2579 flags.go:64] FLAG: --v="2" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685794 2579 flags.go:64] FLAG: --version="false" Jul 05 13:18:15.693358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685798 2579 flags.go:64] FLAG: --vmodule="" Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685807 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.685812 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685928 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685932 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685934 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685937 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685940 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685942 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685947 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685950 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685954 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685957 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685960 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685963 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685965 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685969 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685971 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685973 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685976 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:15.693966 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685978 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685981 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685983 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685986 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685988 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685992 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685995 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.685998 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686001 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686003 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686006 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686008 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686011 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686013 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686024 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686027 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686030 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686032 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686034 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:15.694439 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686037 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686039 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686041 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686044 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686047 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686050 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686052 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686055 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686057 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686060 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686062 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686065 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686067 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686069 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686072 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686074 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686077 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686079 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686082 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686084 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:15.694912 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686086 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686089 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686091 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686094 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686096 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686099 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686101 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686105 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686113 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686115 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686118 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686120 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686122 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686125 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686127 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686130 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686134 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686137 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686139 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686142 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:15.695382 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686144 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686146 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686149 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686151 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686154 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686157 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686160 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686163 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686166 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686169 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686171 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686174 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686176 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686179 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686182 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686184 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686186 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686189 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686191 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:15.695952 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686195 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686197 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686205 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.686208 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.686969 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.693661 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.693689 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.693709 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.693715 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693781 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693786 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693790 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693794 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693797 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:15.696416 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693800 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693803 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693806 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693808 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693811 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693813 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693816 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693818 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693820 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693823 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693825 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693828 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693830 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693833 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693835 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693838 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693840 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693843 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693846 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:15.696797 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693848 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693850 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693853 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693855 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693857 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693861 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693864 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693866 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693869 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693871 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693874 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693876 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693879 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693881 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693883 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693886 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693888 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693890 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693893 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:15.697253 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693895 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693898 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693900 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693902 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693905 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693907 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693910 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693912 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693914 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693916 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693919 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693921 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693924 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693927 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693931 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693933 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693935 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693938 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693941 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693944 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:15.697722 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693947 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693949 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693952 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693954 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693957 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693959 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693961 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693964 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693966 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693970 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693974 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693976 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693979 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693982 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693985 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693987 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693990 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693992 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693995 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.693997 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:15.698200 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694000 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694002 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694004 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694007 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694010 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694012 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694014 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694017 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694019 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694022 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694025 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694028 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694031 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694034 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694036 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:15.698701 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694039 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.694044 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694150 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694153 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694156 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694159 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694162 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694166 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694169 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694172 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694174 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694177 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694179 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694182 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694184 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 13:18:15.699067 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694186 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694189 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694191 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694194 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694196 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694198 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694201 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694203 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694205 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694208 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694210 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694213 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694216 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694218 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694221 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694224 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694226 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694229 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694231 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694234 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 13:18:15.699440 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694236 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694238 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694241 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694243 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694245 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694248 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694250 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694252 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694255 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694258 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694262 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694264 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694267 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694270 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694273 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694275 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694278 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694280 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694283 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694285 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 13:18:15.700000 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694287 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694290 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694292 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694295 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694298 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694301 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694303 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694305 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694308 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694312 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694315 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694318 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694321 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694323 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694326 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694328 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694331 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694334 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694337 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 13:18:15.700479 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694339 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694341 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694344 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694347 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694349 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694351 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694354 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694356 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694358 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694361 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694363 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694366 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694368 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694370 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694373 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694375 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694377 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694380 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694383 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694385 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 13:18:15.700959 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694388 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694390 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694393 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694395 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694398 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694400 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:15.694403 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.694407 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.694500 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.696532 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.698067 2579 server.go:1019] "Starting client certificate rotation" Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.698165 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.698200 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 13:18:15.701423 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.700891 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:18:15.704642 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.704626 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 13:18:15.714740 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.714722 2579 log.go:25] "Validated CRI v1 runtime API" Jul 05 13:18:15.717466 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.717449 2579 log.go:25] "Validated CRI v1 image API" Jul 05 13:18:15.718333 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.718318 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 05 13:18:15.722157 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.722137 2579 fs.go:135] Filesystem UUIDs: map[709fc3f4-d777-4183-908e-67ded06159da:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 dd7df768-a42b-489d-8a60-13bc960b5a1d:/dev/nvme0n1p3] Jul 05 13:18:15.722230 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.722157 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 05 13:18:15.728888 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.728783 2579 manager.go:216] Machine: {Timestamp:2026-07-05 13:18:15.727420498 +0000 UTC m=+0.355786244 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3121215 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec207fb903a9068bfaf03689439ea16a SystemUUID:ec207fb9-03a9-068b-faf0-3689439ea16a BootID:954de147-6561-4e40-8972-8790fa00b323 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:34:de:f8:c9:d7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:34:de:f8:c9:d7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:9a:09:17:e9:e3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 05 13:18:15.728888 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.728882 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 05 13:18:15.729062 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.728973 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 05 13:18:15.731186 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.731168 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:18:15.732953 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.732928 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 05 13:18:15.733084 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.732957 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-21.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 05 13:18:15.733663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.733653 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 05 13:18:15.733713 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.733666 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 05 13:18:15.733713 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.733690 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:18:15.734286 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.734277 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 13:18:15.735591 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.735582 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 05 13:18:15.739642 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.739630 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:18:15.739783 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.739773 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 05 13:18:15.739814 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.739806 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 05 13:18:15.739841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.739815 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 05 13:18:15.739841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.739826 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 05 13:18:15.739841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.739833 2579 kubelet.go:398] "Adding apiserver pod source" Jul 05 13:18:15.739841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.739841 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 05 13:18:15.740178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.740167 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:18:15.740224 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.740187 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 13:18:15.742386 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.742372 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 05 13:18:15.744191 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.744178 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 05 13:18:15.744231 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.744208 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 05 13:18:15.745474 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745462 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 05 13:18:15.745474 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745477 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745483 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745488 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745497 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745514 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745522 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745527 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745533 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745538 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745553 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 05 13:18:15.745573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.745558 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 05 13:18:15.747012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.747002 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 05 13:18:15.747012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.747013 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 05 13:18:15.750213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.750197 2579 server.go:1268] "Started kubelet" Jul 05 13:18:15.750310 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.750265 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 05 13:18:15.750310 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.750272 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 05 13:18:15.750421 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.750364 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 05 13:18:15.751073 ip-10-0-143-21 systemd[1]: Started Kubernetes Kubelet. Jul 05 13:18:15.752055 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.751906 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 05 13:18:15.752222 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.752206 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 05 13:18:15.756453 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.756425 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 05 13:18:15.759829 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.759796 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-21.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 05 13:18:15.759930 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.759871 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-21.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 05 13:18:15.759984 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.759939 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 05 13:18:15.760748 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.760723 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 05 13:18:15.760748 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.760725 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 05 13:18:15.760875 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.760809 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:15.760875 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.759813 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-21.ec2.internal.18bf673e893dd9cb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-21.ec2.internal,UID:ip-10-0-143-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-21.ec2.internal,},FirstTimestamp:2026-07-05 13:18:15.750171083 +0000 UTC m=+0.378536828,LastTimestamp:2026-07-05 13:18:15.750171083 +0000 UTC m=+0.378536828,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-21.ec2.internal,}" Jul 05 13:18:15.761382 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761351 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 05 13:18:15.761382 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761376 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 05 13:18:15.761382 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761356 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 05 13:18:15.761541 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761452 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 05 13:18:15.761541 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761460 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 05 13:18:15.761636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761598 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 05 13:18:15.761636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761613 2579 factory.go:55] Registering systemd factory Jul 05 13:18:15.761636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761622 2579 factory.go:223] Registration of the systemd container factory successfully Jul 05 13:18:15.761836 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761821 2579 factory.go:153] Registering CRI-O factory Jul 05 13:18:15.761836 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761835 2579 factory.go:223] Registration of the crio container factory successfully Jul 05 13:18:15.762003 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761851 2579 factory.go:103] Registering Raw factory Jul 05 13:18:15.762003 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.761866 2579 manager.go:1196] Started watching for new ooms in manager Jul 05 13:18:15.762350 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.762335 2579 manager.go:349] Starting recovery of all containers Jul 05 13:18:15.770177 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.770030 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 05 13:18:15.770177 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.770119 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-21.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 05 13:18:15.771939 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.771924 2579 manager.go:354] Recovery completed Jul 05 13:18:15.773753 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.773725 2579 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 05 13:18:15.776720 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.776705 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:15.780278 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.780253 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:15.780362 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.780293 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:15.780362 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.780307 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:15.780868 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.780856 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 05 13:18:15.780868 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.780867 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 05 13:18:15.780957 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.780878 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:15.780957 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.780881 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 05 13:18:15.781604 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.781591 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:15.781658 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.781634 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:15.781658 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.781644 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:15.782373 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.782314 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-21.ec2.internal.18bf673e8b093c6c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-21.ec2.internal,UID:ip-10-0-143-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-21.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-21.ec2.internal,},FirstTimestamp:2026-07-05 13:18:15.780277356 +0000 UTC m=+0.408643106,LastTimestamp:2026-07-05 13:18:15.780277356 +0000 UTC m=+0.408643106,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-21.ec2.internal,}" Jul 05 13:18:15.782934 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.782911 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mwzmv" Jul 05 13:18:15.783054 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.783038 2579 policy_none.go:49] "None policy: Start" Jul 05 13:18:15.783130 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.783058 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 05 13:18:15.783130 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.783078 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 05 13:18:15.785666 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.785652 2579 policy_none.go:47] "Start" Jul 05 13:18:15.791528 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.791515 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mwzmv" Jul 05 13:18:15.793567 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.793511 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-21.ec2.internal.18bf673e8b0992a5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-21.ec2.internal,UID:ip-10-0-143-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-21.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-21.ec2.internal,},FirstTimestamp:2026-07-05 13:18:15.780299429 +0000 UTC m=+0.408665175,LastTimestamp:2026-07-05 13:18:15.780299429 +0000 UTC m=+0.408665175,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-21.ec2.internal,}" Jul 05 13:18:15.795572 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.795519 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-21.ec2.internal.18bf673e8b09c55e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-21.ec2.internal,UID:ip-10-0-143-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-21.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-21.ec2.internal,},FirstTimestamp:2026-07-05 13:18:15.780312414 +0000 UTC m=+0.408678160,LastTimestamp:2026-07-05 13:18:15.780312414 +0000 UTC m=+0.408678160,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-21.ec2.internal,}" Jul 05 13:18:15.826199 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.826179 2579 manager.go:337] "Starting Device Plugin manager" Jul 05 13:18:15.826275 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.826232 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 05 13:18:15.826275 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.826242 2579 server.go:87] "Starting device plugin registration server" Jul 05 13:18:15.826485 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.826455 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 05 13:18:15.826538 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.826488 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 05 13:18:15.826581 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.826564 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 05 13:18:15.826653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.826638 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 05 13:18:15.826712 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.826653 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 05 13:18:15.827226 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.827202 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 05 13:18:15.827314 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.827259 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:15.885835 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.885807 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 05 13:18:15.887792 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.887083 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 05 13:18:15.887792 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.887112 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 05 13:18:15.887792 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.887128 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 05 13:18:15.887792 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.887163 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 05 13:18:15.889165 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.889148 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:15.927191 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.927131 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:15.927951 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.927935 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:15.928025 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.927968 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:15.928025 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.927980 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:15.928025 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.928001 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-21.ec2.internal" Jul 05 13:18:15.937131 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.937114 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-21.ec2.internal" Jul 05 13:18:15.937198 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.937133 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-21.ec2.internal\": node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:15.964840 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:15.964819 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:15.987431 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.987408 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal"] Jul 05 13:18:15.987512 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.987501 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:15.988995 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.988979 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:15.989070 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.989007 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:15.989070 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.989016 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:15.991231 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.991219 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:15.991359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.991346 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" Jul 05 13:18:15.991404 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.991374 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:15.991894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.991875 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:15.991978 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.991900 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:15.991978 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.991879 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:15.991978 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.991928 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:15.991978 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.991939 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:15.991978 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.991909 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:15.994618 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.994603 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal" Jul 05 13:18:15.994672 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.994630 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:15.995285 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.995269 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:15.995368 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.995300 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:15.995368 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:15.995314 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:16.009181 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.009163 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-21.ec2.internal\" not found" node="ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.013452 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.013437 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-21.ec2.internal\" not found" node="ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.063732 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.063701 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33d35a027521fec9443ee23f19c336c8-config\") pod \"kube-apiserver-proxy-ip-10-0-143-21.ec2.internal\" (UID: \"33d35a027521fec9443ee23f19c336c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.063732 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.063733 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2699490ce633249cba9277b6b3cebee8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal\" (UID: \"2699490ce633249cba9277b6b3cebee8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.063844 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.063748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2699490ce633249cba9277b6b3cebee8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal\" (UID: \"2699490ce633249cba9277b6b3cebee8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.065797 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.065782 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.164356 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.164319 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2699490ce633249cba9277b6b3cebee8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal\" (UID: \"2699490ce633249cba9277b6b3cebee8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.164356 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.164357 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2699490ce633249cba9277b6b3cebee8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal\" (UID: \"2699490ce633249cba9277b6b3cebee8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.164564 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.164383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33d35a027521fec9443ee23f19c336c8-config\") pod \"kube-apiserver-proxy-ip-10-0-143-21.ec2.internal\" (UID: \"33d35a027521fec9443ee23f19c336c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.164564 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.164434 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33d35a027521fec9443ee23f19c336c8-config\") pod \"kube-apiserver-proxy-ip-10-0-143-21.ec2.internal\" (UID: \"33d35a027521fec9443ee23f19c336c8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.164564 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.164435 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2699490ce633249cba9277b6b3cebee8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal\" (UID: \"2699490ce633249cba9277b6b3cebee8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.164564 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.164435 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2699490ce633249cba9277b6b3cebee8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal\" (UID: \"2699490ce633249cba9277b6b3cebee8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.166446 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.166428 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.267196 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.267146 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.312330 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.312300 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.317333 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.317194 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal" Jul 05 13:18:16.367951 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.367926 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.468472 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.468450 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.568956 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.568930 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.669576 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.669551 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.697737 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.697713 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 05 13:18:16.698181 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.697846 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 13:18:16.770514 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.770489 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.794852 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.794817 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-04 13:13:16 +0000 UTC" deadline="2027-12-05 17:14:27.632740546 +0000 UTC" Jul 05 13:18:16.794972 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.794855 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12435h56m10.837891535s" Jul 05 13:18:16.871510 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.871454 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.893800 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.893776 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 13:18:16.894824 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.894803 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientMemory" Jul 05 13:18:16.894938 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.894844 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 13:18:16.894938 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.894860 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeHasSufficientPID" Jul 05 13:18:16.910465 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:16.910442 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2699490ce633249cba9277b6b3cebee8.slice/crio-47704d0dbb8367805c9f3f8a70d264a45bdd36d74c66f70ce08fb9392ed16fb0 WatchSource:0}: Error finding container 47704d0dbb8367805c9f3f8a70d264a45bdd36d74c66f70ce08fb9392ed16fb0: Status 404 returned error can't find the container with id 47704d0dbb8367805c9f3f8a70d264a45bdd36d74c66f70ce08fb9392ed16fb0 Jul 05 13:18:16.910896 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:16.910876 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d35a027521fec9443ee23f19c336c8.slice/crio-a89d490706b3aa202fb7864ab45f8a0d28bbd7ff4018b2655e92e4453e26fb19 WatchSource:0}: Error finding container a89d490706b3aa202fb7864ab45f8a0d28bbd7ff4018b2655e92e4453e26fb19: Status 404 returned error can't find the container with id a89d490706b3aa202fb7864ab45f8a0d28bbd7ff4018b2655e92e4453e26fb19 Jul 05 13:18:16.915148 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.915135 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:18:16.972037 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:16.972013 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:16.972973 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:16.972956 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:17.059941 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.059917 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:17.072979 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:17.072961 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-21.ec2.internal\" not found" Jul 05 13:18:17.163390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.163335 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:17.261460 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.261435 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" Jul 05 13:18:17.279408 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.279389 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:18:17.280570 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.280557 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal" Jul 05 13:18:17.295089 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.295071 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 13:18:17.741693 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.741643 2579 apiserver.go:52] "Watching apiserver" Jul 05 13:18:17.750301 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.750271 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 05 13:18:17.750771 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.750741 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-sk68q","openshift-multus/multus-additional-cni-plugins-2fr86","openshift-multus/multus-djh4v","openshift-multus/network-metrics-daemon-29f8w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx","openshift-dns/node-resolver-zpbbv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal","openshift-network-diagnostics/network-check-target-6j6fs","openshift-network-operator/iptables-alerter-qtf6g","openshift-ovn-kubernetes/ovnkube-node-zpj2z","kube-system/global-pull-secret-syncer-h4vlk","kube-system/konnectivity-agent-h9dvp","kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal","openshift-cluster-node-tuning-operator/tuned-pxxxq"] Jul 05 13:18:17.753389 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.753365 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.755959 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.755940 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.756633 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.756607 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sw6cd\"" Jul 05 13:18:17.756760 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.756653 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 05 13:18:17.756849 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.756614 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 05 13:18:17.756911 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.756883 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 05 13:18:17.758898 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.758877 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.759231 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.759007 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:18:17.759231 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.759019 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 05 13:18:17.759231 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.759212 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v9wqg\"" Jul 05 13:18:17.761160 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.761140 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:17.761267 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:17.761228 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:17.762558 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.761942 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 05 13:18:17.763227 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.762846 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 05 13:18:17.763227 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.762918 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r77d2\"" Jul 05 13:18:17.763227 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.762994 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 05 13:18:17.763227 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.763087 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 05 13:18:17.763227 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.763202 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 05 13:18:17.763591 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.763575 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.765971 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.765949 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 05 13:18:17.766101 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.766084 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:17.766587 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.766242 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 05 13:18:17.766587 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.766477 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 05 13:18:17.766587 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.766523 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hhpx6\"" Jul 05 13:18:17.767317 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.767294 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 05 13:18:17.767401 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.767326 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 05 13:18:17.767524 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.767500 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 05 13:18:17.768884 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.768864 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 05 13:18:17.769096 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.769076 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 05 13:18:17.769164 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.769097 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:17.769222 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:17.769181 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:17.769336 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.769315 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d5828\"" Jul 05 13:18:17.773676 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.771842 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:17.773676 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.772259 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 13:18:17.775227 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.774944 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 05 13:18:17.775227 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.775000 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:18:17.775227 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.775001 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gv9ch\"" Jul 05 13:18:17.775227 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.775082 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 05 13:18:17.776390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776368 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-modprobe-d\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.776484 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776408 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-var-lib-kubelet\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.776484 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776435 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-var-lib-cni-multus\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.776653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776492 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-slash\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.776653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776520 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-run-netns\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.776653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776549 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-run-multus-certs\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.776653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776572 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-run-openvswitch\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.776653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776587 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/77336325-afeb-4e05-8d32-01d6f60f82e7-ovn-node-metrics-cert\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.776653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776606 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-system-cni-dir\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.776653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776630 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-kubelet\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776726 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/77336325-afeb-4e05-8d32-01d6f60f82e7-env-overrides\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776776 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-run-netns\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776807 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-etc-openvswitch\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776833 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-cni-bin\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776856 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-sysctl-conf\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776892 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-cni-dir\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776922 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-cnibin\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.776973 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-os-release\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.777013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sdh6\" (UniqueName: \"kubernetes.io/projected/373998a7-e4cd-48b0-96fa-a4962a0fbe29-kube-api-access-8sdh6\") pod \"node-ca-sk68q\" (UID: \"373998a7-e4cd-48b0-96fa-a4962a0fbe29\") " pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.777044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.777036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-cni-binary-copy\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.777545 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.777060 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-socket-dir-parent\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.779674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8kq2\" (UniqueName: \"kubernetes.io/projected/6cd26735-59f2-404b-981e-a44acd0f4df9-kube-api-access-k8kq2\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.779773 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-run-ovn\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.779885 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.779964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-kubernetes\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780106 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780118 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-var-lib-cni-bin\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780150 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shpts\" (UniqueName: \"kubernetes.io/projected/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-kube-api-access-shpts\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780451 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-systemd-units\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780625 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-run-systemd\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/373998a7-e4cd-48b0-96fa-a4962a0fbe29-serviceca\") pod \"node-ca-sk68q\" (UID: \"373998a7-e4cd-48b0-96fa-a4962a0fbe29\") " pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-hostroot\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780770 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-daemon-config\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780798 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/77336325-afeb-4e05-8d32-01d6f60f82e7-ovnkube-script-lib\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780885 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-sysconfig\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.781037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.780937 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-tuned\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781128 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-var-lib-kubelet\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781208 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-conf-dir\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781261 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5c57816-0ffa-4875-9f54-d56e1e4eed07-tmp\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781293 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781322 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-var-lib-openvswitch\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781368 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-sys\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781399 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-log-socket\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781423 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-cni-netd\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/373998a7-e4cd-48b0-96fa-a4962a0fbe29-host\") pod \"node-ca-sk68q\" (UID: \"373998a7-e4cd-48b0-96fa-a4962a0fbe29\") " pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781565 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-host\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781595 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-node-log\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781633 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-run-k8s-cni-cncf-io\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781668 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-sysctl-d\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781716 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-systemd\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781765 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-lib-modules\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.781928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781796 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgwtx\" (UniqueName: \"kubernetes.io/projected/77336325-afeb-4e05-8d32-01d6f60f82e7-kube-api-access-pgwtx\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.782871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781828 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-run\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.782871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781858 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98mft\" (UniqueName: \"kubernetes.io/projected/d5c57816-0ffa-4875-9f54-d56e1e4eed07-kube-api-access-98mft\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.782871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781888 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-etc-kubernetes\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.782871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.781921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/77336325-afeb-4e05-8d32-01d6f60f82e7-ovnkube-config\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.782871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.782188 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:17.782871 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:17.782269 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:17.783435 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.783385 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 05 13:18:17.783684 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.783664 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 05 13:18:17.783897 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.783823 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nt6j6\"" Jul 05 13:18:17.784370 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.784346 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 05 13:18:17.785549 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.785524 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:17.788243 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.788160 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.788996 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.788677 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 05 13:18:17.788996 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.788680 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 05 13:18:17.788996 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.788916 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-d6dmn\"" Jul 05 13:18:17.790960 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.790938 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 05 13:18:17.791040 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.790959 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2g7q2\"" Jul 05 13:18:17.791232 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.791213 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 05 13:18:17.805935 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.805913 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mq7jd" Jul 05 13:18:17.816784 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.816761 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mq7jd" Jul 05 13:18:17.863290 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.863265 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 05 13:18:17.883046 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883014 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-run-netns\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883064 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e7cb017-f290-456f-8c62-a896ceb8e01e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.883178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883093 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77r7z\" (UniqueName: \"kubernetes.io/projected/3e7cb017-f290-456f-8c62-a896ceb8e01e-kube-api-access-77r7z\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.883178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883120 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-run-multus-certs\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-run-openvswitch\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/77336325-afeb-4e05-8d32-01d6f60f82e7-ovn-node-metrics-cert\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-system-cni-dir\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883193 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-kubelet\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883209 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/77336325-afeb-4e05-8d32-01d6f60f82e7-env-overrides\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883223 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-device-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883257 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-dbus\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883281 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883304 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e7cb017-f290-456f-8c62-a896ceb8e01e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883331 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-run-netns\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883361 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-etc-openvswitch\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883384 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-cni-bin\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883416 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-sysctl-conf\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883451 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-cni-dir\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883479 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-cnibin\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883495 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-os-release\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883515 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb9ca114-8751-44b3-b13a-713cf3f44f18-konnectivity-ca\") pod \"konnectivity-agent-h9dvp\" (UID: \"cb9ca114-8751-44b3-b13a-713cf3f44f18\") " pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883532 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883569 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdh6\" (UniqueName: \"kubernetes.io/projected/373998a7-e4cd-48b0-96fa-a4962a0fbe29-kube-api-access-8sdh6\") pod \"node-ca-sk68q\" (UID: \"373998a7-e4cd-48b0-96fa-a4962a0fbe29\") " pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883587 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-cni-binary-copy\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883612 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-socket-dir-parent\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883654 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8kq2\" (UniqueName: \"kubernetes.io/projected/6cd26735-59f2-404b-981e-a44acd0f4df9-kube-api-access-k8kq2\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883680 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-run-ovn\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883726 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-kubernetes\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883796 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-var-lib-cni-bin\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883823 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shpts\" (UniqueName: \"kubernetes.io/projected/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-kube-api-access-shpts\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883845 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-systemd-units\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-run-systemd\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883891 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/373998a7-e4cd-48b0-96fa-a4962a0fbe29-serviceca\") pod \"node-ca-sk68q\" (UID: \"373998a7-e4cd-48b0-96fa-a4962a0fbe29\") " pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883914 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-hostroot\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.883979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883930 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-daemon-config\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/77336325-afeb-4e05-8d32-01d6f60f82e7-ovnkube-script-lib\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883963 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.883979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-kubelet-config\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884005 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884032 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-sysconfig\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884072 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-tuned\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884096 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-cnibin\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884150 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-run-netns\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884170 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-os-release\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884212 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-var-lib-kubelet\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884264 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-run-openvswitch\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.884716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884654 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885006 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-cni-binary-copy\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.884098 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-var-lib-kubelet\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885117 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-kubernetes\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885128 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-system-cni-dir\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885161 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-conf-dir\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885169 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-cni-bin\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885119 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-conf-dir\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885156 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-var-lib-cni-bin\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885176 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-kubelet\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885117 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-hostroot\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885212 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-run-ovn\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885236 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-run-netns\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885261 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885269 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/373998a7-e4cd-48b0-96fa-a4962a0fbe29-serviceca\") pod \"node-ca-sk68q\" (UID: \"373998a7-e4cd-48b0-96fa-a4962a0fbe29\") " pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.885316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885304 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-socket-dir-parent\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885458 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-etc-openvswitch\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885481 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/77336325-afeb-4e05-8d32-01d6f60f82e7-env-overrides\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885724 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-run-systemd\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885770 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-run-multus-certs\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885823 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-sysconfig\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885854 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-systemd-units\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885890 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-cni-dir\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.885985 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-sysctl-conf\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886023 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-registration-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886053 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-multus-daemon-config\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886057 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3f358ef2-7b32-47c0-aaac-5c1048a34441-hosts-file\") pod \"node-resolver-zpbbv\" (UID: \"3f358ef2-7b32-47c0-aaac-5c1048a34441\") " pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886101 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5c57816-0ffa-4875-9f54-d56e1e4eed07-tmp\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.886213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886155 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886265 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-var-lib-openvswitch\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886374 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kczlw\" (UniqueName: \"kubernetes.io/projected/3f358ef2-7b32-47c0-aaac-5c1048a34441-kube-api-access-kczlw\") pod \"node-resolver-zpbbv\" (UID: \"3f358ef2-7b32-47c0-aaac-5c1048a34441\") " pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:17.886404 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886417 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-var-lib-openvswitch\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886467 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-sys\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886480 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/77336325-afeb-4e05-8d32-01d6f60f82e7-ovnkube-script-lib\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:17.886529 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs podName:6cd26735-59f2-404b-981e-a44acd0f4df9 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:18.386506582 +0000 UTC m=+3.014872348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs") pod "network-metrics-daemon-29f8w" (UID: "6cd26735-59f2-404b-981e-a44acd0f4df9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886534 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-sys\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886551 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-log-socket\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886590 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886604 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-log-socket\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886624 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-cni-netd\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886642 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886655 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmgm6\" (UniqueName: \"kubernetes.io/projected/6659e427-5fd8-4cae-9675-4900eef21c94-kube-api-access-pmgm6\") pod \"iptables-alerter-qtf6g\" (UID: \"6659e427-5fd8-4cae-9675-4900eef21c94\") " pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886699 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-socket-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886707 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-cni-netd\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.886927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886759 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/373998a7-e4cd-48b0-96fa-a4962a0fbe29-host\") pod \"node-ca-sk68q\" (UID: \"373998a7-e4cd-48b0-96fa-a4962a0fbe29\") " pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886785 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-host\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886807 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-node-log\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886833 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6659e427-5fd8-4cae-9675-4900eef21c94-iptables-alerter-script\") pod \"iptables-alerter-qtf6g\" (UID: \"6659e427-5fd8-4cae-9675-4900eef21c94\") " pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886848 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-host\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886858 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-os-release\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886848 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/373998a7-e4cd-48b0-96fa-a4962a0fbe29-host\") pod \"node-ca-sk68q\" (UID: \"373998a7-e4cd-48b0-96fa-a4962a0fbe29\") " pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886883 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-node-log\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-run-k8s-cni-cncf-io\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886919 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-sysctl-d\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886933 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-run-k8s-cni-cncf-io\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886938 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnnnd\" (UniqueName: \"kubernetes.io/projected/175a21cc-711e-420f-8ea9-80011c45fbd5-kube-api-access-tnnnd\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886980 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-systemd\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.886996 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-lib-modules\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887014 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgwtx\" (UniqueName: \"kubernetes.io/projected/77336325-afeb-4e05-8d32-01d6f60f82e7-kube-api-access-pgwtx\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887040 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-sysctl-d\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887041 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.887843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887087 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f358ef2-7b32-47c0-aaac-5c1048a34441-tmp-dir\") pod \"node-resolver-zpbbv\" (UID: \"3f358ef2-7b32-47c0-aaac-5c1048a34441\") " pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887084 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-systemd\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-cnibin\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887152 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-run\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887178 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98mft\" (UniqueName: \"kubernetes.io/projected/d5c57816-0ffa-4875-9f54-d56e1e4eed07-kube-api-access-98mft\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887183 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-lib-modules\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887204 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-etc-kubernetes\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887237 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-etc-kubernetes\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887255 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/77336325-afeb-4e05-8d32-01d6f60f82e7-ovnkube-config\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887286 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-run\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887285 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6659e427-5fd8-4cae-9675-4900eef21c94-host-slash\") pod \"iptables-alerter-qtf6g\" (UID: \"6659e427-5fd8-4cae-9675-4900eef21c94\") " pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887323 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-sys-fs\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887350 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb9ca114-8751-44b3-b13a-713cf3f44f18-agent-certs\") pod \"konnectivity-agent-h9dvp\" (UID: \"cb9ca114-8751-44b3-b13a-713cf3f44f18\") " pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887375 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e7cb017-f290-456f-8c62-a896ceb8e01e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-modprobe-d\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887430 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-var-lib-kubelet\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887458 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-var-lib-cni-multus\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.888635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887486 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-slash\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.889475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887520 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-system-cni-dir\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.889475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887574 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-var-lib-kubelet\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.889475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887619 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-host-var-lib-cni-multus\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.889475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887624 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/77336325-afeb-4e05-8d32-01d6f60f82e7-host-slash\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.889475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887707 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-modprobe-d\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.889475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.887798 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/77336325-afeb-4e05-8d32-01d6f60f82e7-ovnkube-config\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.889475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.888662 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5c57816-0ffa-4875-9f54-d56e1e4eed07-tmp\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.889475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.888708 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5c57816-0ffa-4875-9f54-d56e1e4eed07-etc-tuned\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.889475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.888783 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/77336325-afeb-4e05-8d32-01d6f60f82e7-ovn-node-metrics-cert\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.892374 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.892258 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal" event={"ID":"33d35a027521fec9443ee23f19c336c8","Type":"ContainerStarted","Data":"a89d490706b3aa202fb7864ab45f8a0d28bbd7ff4018b2655e92e4453e26fb19"} Jul 05 13:18:17.893674 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.893647 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" event={"ID":"2699490ce633249cba9277b6b3cebee8","Type":"ContainerStarted","Data":"47704d0dbb8367805c9f3f8a70d264a45bdd36d74c66f70ce08fb9392ed16fb0"} Jul 05 13:18:17.901567 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.901499 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8kq2\" (UniqueName: \"kubernetes.io/projected/6cd26735-59f2-404b-981e-a44acd0f4df9-kube-api-access-k8kq2\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:17.901567 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.901523 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgwtx\" (UniqueName: \"kubernetes.io/projected/77336325-afeb-4e05-8d32-01d6f60f82e7-kube-api-access-pgwtx\") pod \"ovnkube-node-zpj2z\" (UID: \"77336325-afeb-4e05-8d32-01d6f60f82e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:17.903672 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.903647 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shpts\" (UniqueName: \"kubernetes.io/projected/4516d9bd-dd1b-4141-b7af-b1f2f2107ec5-kube-api-access-shpts\") pod \"multus-djh4v\" (UID: \"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5\") " pod="openshift-multus/multus-djh4v" Jul 05 13:18:17.905829 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.905800 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98mft\" (UniqueName: \"kubernetes.io/projected/d5c57816-0ffa-4875-9f54-d56e1e4eed07-kube-api-access-98mft\") pod \"tuned-pxxxq\" (UID: \"d5c57816-0ffa-4875-9f54-d56e1e4eed07\") " pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:17.910165 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.910111 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sdh6\" (UniqueName: \"kubernetes.io/projected/373998a7-e4cd-48b0-96fa-a4962a0fbe29-kube-api-access-8sdh6\") pod \"node-ca-sk68q\" (UID: \"373998a7-e4cd-48b0-96fa-a4962a0fbe29\") " pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:17.988205 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988169 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988218 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-kubelet-config\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988248 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-registration-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3f358ef2-7b32-47c0-aaac-5c1048a34441-hosts-file\") pod \"node-resolver-zpbbv\" (UID: \"3f358ef2-7b32-47c0-aaac-5c1048a34441\") " pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988318 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-kubelet-config\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988318 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988339 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kczlw\" (UniqueName: \"kubernetes.io/projected/3f358ef2-7b32-47c0-aaac-5c1048a34441-kube-api-access-kczlw\") pod \"node-resolver-zpbbv\" (UID: \"3f358ef2-7b32-47c0-aaac-5c1048a34441\") " pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988358 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmgm6\" (UniqueName: \"kubernetes.io/projected/6659e427-5fd8-4cae-9675-4900eef21c94-kube-api-access-pmgm6\") pod \"iptables-alerter-qtf6g\" (UID: \"6659e427-5fd8-4cae-9675-4900eef21c94\") " pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988374 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-socket-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988375 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-registration-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.988394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988391 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6659e427-5fd8-4cae-9675-4900eef21c94-iptables-alerter-script\") pod \"iptables-alerter-qtf6g\" (UID: \"6659e427-5fd8-4cae-9675-4900eef21c94\") " pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988413 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-os-release\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988439 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnnnd\" (UniqueName: \"kubernetes.io/projected/175a21cc-711e-420f-8ea9-80011c45fbd5-kube-api-access-tnnnd\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988457 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3f358ef2-7b32-47c0-aaac-5c1048a34441-hosts-file\") pod \"node-resolver-zpbbv\" (UID: \"3f358ef2-7b32-47c0-aaac-5c1048a34441\") " pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988493 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f358ef2-7b32-47c0-aaac-5c1048a34441-tmp-dir\") pod \"node-resolver-zpbbv\" (UID: \"3f358ef2-7b32-47c0-aaac-5c1048a34441\") " pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988503 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988516 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-cnibin\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988565 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6659e427-5fd8-4cae-9675-4900eef21c94-host-slash\") pod \"iptables-alerter-qtf6g\" (UID: \"6659e427-5fd8-4cae-9675-4900eef21c94\") " pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988567 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-socket-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988570 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-cnibin\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988628 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6659e427-5fd8-4cae-9675-4900eef21c94-host-slash\") pod \"iptables-alerter-qtf6g\" (UID: \"6659e427-5fd8-4cae-9675-4900eef21c94\") " pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988658 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-sys-fs\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988686 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb9ca114-8751-44b3-b13a-713cf3f44f18-agent-certs\") pod \"konnectivity-agent-h9dvp\" (UID: \"cb9ca114-8751-44b3-b13a-713cf3f44f18\") " pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988730 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e7cb017-f290-456f-8c62-a896ceb8e01e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988763 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-system-cni-dir\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988776 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-sys-fs\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.989082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988791 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e7cb017-f290-456f-8c62-a896ceb8e01e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988819 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77r7z\" (UniqueName: \"kubernetes.io/projected/3e7cb017-f290-456f-8c62-a896ceb8e01e-kube-api-access-77r7z\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988850 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-device-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988873 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-dbus\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988890 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f358ef2-7b32-47c0-aaac-5c1048a34441-tmp-dir\") pod \"node-resolver-zpbbv\" (UID: \"3f358ef2-7b32-47c0-aaac-5c1048a34441\") " pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988897 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988905 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-etc-selinux\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.988923 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e7cb017-f290-456f-8c62-a896ceb8e01e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989117 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6659e427-5fd8-4cae-9675-4900eef21c94-iptables-alerter-script\") pod \"iptables-alerter-qtf6g\" (UID: \"6659e427-5fd8-4cae-9675-4900eef21c94\") " pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989282 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-dbus\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989338 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/175a21cc-711e-420f-8ea9-80011c45fbd5-device-dir\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989396 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-os-release\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989427 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb9ca114-8751-44b3-b13a-713cf3f44f18-konnectivity-ca\") pod \"konnectivity-agent-h9dvp\" (UID: \"cb9ca114-8751-44b3-b13a-713cf3f44f18\") " pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989489 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e7cb017-f290-456f-8c62-a896ceb8e01e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989539 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e7cb017-f290-456f-8c62-a896ceb8e01e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989552 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e7cb017-f290-456f-8c62-a896ceb8e01e-system-cni-dir\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989606 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e7cb017-f290-456f-8c62-a896ceb8e01e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:17.989915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.989635 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:17.990891 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:17.989644 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:17.990891 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:17.989696 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret podName:dcb90eb9-329e-4a35-9ced-6d86eb35eb4f nodeName:}" failed. No retries permitted until 2026-07-05 13:18:18.489677901 +0000 UTC m=+3.118043636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret") pod "global-pull-secret-syncer-h4vlk" (UID: "dcb90eb9-329e-4a35-9ced-6d86eb35eb4f") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:17.990891 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.990060 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb9ca114-8751-44b3-b13a-713cf3f44f18-konnectivity-ca\") pod \"konnectivity-agent-h9dvp\" (UID: \"cb9ca114-8751-44b3-b13a-713cf3f44f18\") " pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:17.991639 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:17.991613 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb9ca114-8751-44b3-b13a-713cf3f44f18-agent-certs\") pod \"konnectivity-agent-h9dvp\" (UID: \"cb9ca114-8751-44b3-b13a-713cf3f44f18\") " pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:18.007900 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.007825 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:18.007900 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.007852 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:18.007900 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.007868 2579 projected.go:196] Error preparing data for projected volume kube-api-access-w9thn for pod openshift-network-diagnostics/network-check-target-6j6fs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:18.007900 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.007883 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kczlw\" (UniqueName: \"kubernetes.io/projected/3f358ef2-7b32-47c0-aaac-5c1048a34441-kube-api-access-kczlw\") pod \"node-resolver-zpbbv\" (UID: \"3f358ef2-7b32-47c0-aaac-5c1048a34441\") " pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:18.008119 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.007944 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn podName:2798e10e-cb4f-4dd5-a665-f6fc6712315c nodeName:}" failed. No retries permitted until 2026-07-05 13:18:18.507923868 +0000 UTC m=+3.136289624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w9thn" (UniqueName: "kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn") pod "network-check-target-6j6fs" (UID: "2798e10e-cb4f-4dd5-a665-f6fc6712315c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:18.008170 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.008141 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77r7z\" (UniqueName: \"kubernetes.io/projected/3e7cb017-f290-456f-8c62-a896ceb8e01e-kube-api-access-77r7z\") pod \"multus-additional-cni-plugins-2fr86\" (UID: \"3e7cb017-f290-456f-8c62-a896ceb8e01e\") " pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:18.009505 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.009482 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnnnd\" (UniqueName: \"kubernetes.io/projected/175a21cc-711e-420f-8ea9-80011c45fbd5-kube-api-access-tnnnd\") pod \"aws-ebs-csi-driver-node-ns5bx\" (UID: \"175a21cc-711e-420f-8ea9-80011c45fbd5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:18.012663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.012641 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmgm6\" (UniqueName: \"kubernetes.io/projected/6659e427-5fd8-4cae-9675-4900eef21c94-kube-api-access-pmgm6\") pod \"iptables-alerter-qtf6g\" (UID: \"6659e427-5fd8-4cae-9675-4900eef21c94\") " pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:18.071028 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.070960 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sk68q" Jul 05 13:18:18.081615 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.081594 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" Jul 05 13:18:18.091854 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.091831 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-djh4v" Jul 05 13:18:18.098869 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.098851 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:18.106773 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.106749 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zpbbv" Jul 05 13:18:18.115442 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.115414 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qtf6g" Jul 05 13:18:18.125251 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.125229 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" Jul 05 13:18:18.132146 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.132125 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:18.139848 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.139823 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2fr86" Jul 05 13:18:18.258952 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.258880 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 13:18:18.393204 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.393171 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:18.393394 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.393312 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:18.393394 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.393386 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs podName:6cd26735-59f2-404b-981e-a44acd0f4df9 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:19.393365797 +0000 UTC m=+4.021731549 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs") pod "network-metrics-daemon-29f8w" (UID: "6cd26735-59f2-404b-981e-a44acd0f4df9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:18.494323 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.494288 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:18.494538 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.494462 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:18.494538 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.494536 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret podName:dcb90eb9-329e-4a35-9ced-6d86eb35eb4f nodeName:}" failed. No retries permitted until 2026-07-05 13:18:19.494512539 +0000 UTC m=+4.122878277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret") pod "global-pull-secret-syncer-h4vlk" (UID: "dcb90eb9-329e-4a35-9ced-6d86eb35eb4f") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:18.595697 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.595664 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:18.595891 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.595843 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:18.595891 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.595859 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:18.595891 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.595868 2579 projected.go:196] Error preparing data for projected volume kube-api-access-w9thn for pod openshift-network-diagnostics/network-check-target-6j6fs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:18.596013 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:18.595919 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn podName:2798e10e-cb4f-4dd5-a665-f6fc6712315c nodeName:}" failed. No retries permitted until 2026-07-05 13:18:19.595906198 +0000 UTC m=+4.224271936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9thn" (UniqueName: "kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn") pod "network-check-target-6j6fs" (UID: "2798e10e-cb4f-4dd5-a665-f6fc6712315c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:18.820670 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.820613 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:13:18 +0000 UTC" deadline="2028-01-18 03:50:16.237610422 +0000 UTC" Jul 05 13:18:18.820670 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:18.820662 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13478h31m57.416954541s" Jul 05 13:18:19.036545 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:19.036518 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7cb017_f290_456f_8c62_a896ceb8e01e.slice/crio-3aba69c93c7bd559445864fe7d2e72022a3cc824d2a7f67edd57c7021fb34ab8 WatchSource:0}: Error finding container 3aba69c93c7bd559445864fe7d2e72022a3cc824d2a7f67edd57c7021fb34ab8: Status 404 returned error can't find the container with id 3aba69c93c7bd559445864fe7d2e72022a3cc824d2a7f67edd57c7021fb34ab8 Jul 05 13:18:19.038146 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:19.037985 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373998a7_e4cd_48b0_96fa_a4962a0fbe29.slice/crio-c9d13c153463463240a19d84dcd21e2c47cc3d6e7ab5a9ef60adaf78942e1d26 WatchSource:0}: Error finding container c9d13c153463463240a19d84dcd21e2c47cc3d6e7ab5a9ef60adaf78942e1d26: Status 404 returned error can't find the container with id c9d13c153463463240a19d84dcd21e2c47cc3d6e7ab5a9ef60adaf78942e1d26 Jul 05 13:18:19.040645 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:19.040620 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c57816_0ffa_4875_9f54_d56e1e4eed07.slice/crio-94d207f171081c1e606398e628ecbbb4e4f80b47e2d9c5f39a726b69c62a9285 WatchSource:0}: Error finding container 94d207f171081c1e606398e628ecbbb4e4f80b47e2d9c5f39a726b69c62a9285: Status 404 returned error can't find the container with id 94d207f171081c1e606398e628ecbbb4e4f80b47e2d9c5f39a726b69c62a9285 Jul 05 13:18:19.042770 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:19.042732 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175a21cc_711e_420f_8ea9_80011c45fbd5.slice/crio-1819350e6208b13ee926635c81cdd77fdd1f2813d531dfe3eaf13aa723951582 WatchSource:0}: Error finding container 1819350e6208b13ee926635c81cdd77fdd1f2813d531dfe3eaf13aa723951582: Status 404 returned error can't find the container with id 1819350e6208b13ee926635c81cdd77fdd1f2813d531dfe3eaf13aa723951582 Jul 05 13:18:19.045097 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:19.044891 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77336325_afeb_4e05_8d32_01d6f60f82e7.slice/crio-64f2847c0461ab64e3563ffa36bb45f71f7400e8c7ea6fc601c11459e0ffe11a WatchSource:0}: Error finding container 64f2847c0461ab64e3563ffa36bb45f71f7400e8c7ea6fc601c11459e0ffe11a: Status 404 returned error can't find the container with id 64f2847c0461ab64e3563ffa36bb45f71f7400e8c7ea6fc601c11459e0ffe11a Jul 05 13:18:19.045547 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:19.045410 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9ca114_8751_44b3_b13a_713cf3f44f18.slice/crio-eff634a78eba5499925709e235e5b4370b0e09083a398ea1b409d84ee5708360 WatchSource:0}: Error finding container eff634a78eba5499925709e235e5b4370b0e09083a398ea1b409d84ee5708360: Status 404 returned error can't find the container with id eff634a78eba5499925709e235e5b4370b0e09083a398ea1b409d84ee5708360 Jul 05 13:18:19.047304 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:19.047193 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f358ef2_7b32_47c0_aaac_5c1048a34441.slice/crio-41d8c305ce05e1ab393ac2c899f4a21ea4eb8f0305ae01e7bed469c321a9bf72 WatchSource:0}: Error finding container 41d8c305ce05e1ab393ac2c899f4a21ea4eb8f0305ae01e7bed469c321a9bf72: Status 404 returned error can't find the container with id 41d8c305ce05e1ab393ac2c899f4a21ea4eb8f0305ae01e7bed469c321a9bf72 Jul 05 13:18:19.047818 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:19.047671 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4516d9bd_dd1b_4141_b7af_b1f2f2107ec5.slice/crio-e346fab09aa2ed06b3fb5f84391efa9d444a482581d6443c041497424d828971 WatchSource:0}: Error finding container e346fab09aa2ed06b3fb5f84391efa9d444a482581d6443c041497424d828971: Status 404 returned error can't find the container with id e346fab09aa2ed06b3fb5f84391efa9d444a482581d6443c041497424d828971 Jul 05 13:18:19.050903 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:19.050845 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6659e427_5fd8_4cae_9675_4900eef21c94.slice/crio-61fb71c9bbe929faf157712f0fcc7bd92c768f70dcdf99d21e80cea2314596b4 WatchSource:0}: Error finding container 61fb71c9bbe929faf157712f0fcc7bd92c768f70dcdf99d21e80cea2314596b4: Status 404 returned error can't find the container with id 61fb71c9bbe929faf157712f0fcc7bd92c768f70dcdf99d21e80cea2314596b4 Jul 05 13:18:19.401569 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.401362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:19.401698 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.401520 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:19.401698 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.401662 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs podName:6cd26735-59f2-404b-981e-a44acd0f4df9 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:21.401642317 +0000 UTC m=+6.030008067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs") pod "network-metrics-daemon-29f8w" (UID: "6cd26735-59f2-404b-981e-a44acd0f4df9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:19.502558 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.501892 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:19.502558 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.502092 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:19.502558 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.502157 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret podName:dcb90eb9-329e-4a35-9ced-6d86eb35eb4f nodeName:}" failed. No retries permitted until 2026-07-05 13:18:21.502137808 +0000 UTC m=+6.130503563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret") pod "global-pull-secret-syncer-h4vlk" (UID: "dcb90eb9-329e-4a35-9ced-6d86eb35eb4f") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:19.603099 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.603056 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:19.603294 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.603273 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:19.603362 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.603307 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:19.603362 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.603321 2579 projected.go:196] Error preparing data for projected volume kube-api-access-w9thn for pod openshift-network-diagnostics/network-check-target-6j6fs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:19.603461 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.603383 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn podName:2798e10e-cb4f-4dd5-a665-f6fc6712315c nodeName:}" failed. No retries permitted until 2026-07-05 13:18:21.60336132 +0000 UTC m=+6.231727058 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9thn" (UniqueName: "kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn") pod "network-check-target-6j6fs" (UID: "2798e10e-cb4f-4dd5-a665-f6fc6712315c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:19.820924 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.820853 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 13:13:18 +0000 UTC" deadline="2028-04-05 16:55:48.272169548 +0000 UTC" Jul 05 13:18:19.820924 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.820921 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15363h37m28.451253024s" Jul 05 13:18:19.896788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.896350 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:19.896788 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.896442 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:19.896788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.896509 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:19.896788 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.896563 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:19.896788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.896598 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:19.896788 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:19.896635 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:19.905865 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.905390 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h9dvp" event={"ID":"cb9ca114-8751-44b3-b13a-713cf3f44f18","Type":"ContainerStarted","Data":"eff634a78eba5499925709e235e5b4370b0e09083a398ea1b409d84ee5708360"} Jul 05 13:18:19.934773 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.929040 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" event={"ID":"d5c57816-0ffa-4875-9f54-d56e1e4eed07","Type":"ContainerStarted","Data":"94d207f171081c1e606398e628ecbbb4e4f80b47e2d9c5f39a726b69c62a9285"} Jul 05 13:18:19.939151 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.939119 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal" event={"ID":"33d35a027521fec9443ee23f19c336c8","Type":"ContainerStarted","Data":"1fc4dcc62d85b427ac733927eb64058125d106760c0d236a66f254f6e5a3fa12"} Jul 05 13:18:19.951985 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.951955 2579 generic.go:356] "Generic (PLEG): container finished" podID="2699490ce633249cba9277b6b3cebee8" containerID="3cc7a1a8a0ea50669985b78d51a93ce30caa0f4896b51f05aae5d582b8f788d6" exitCode=0 Jul 05 13:18:19.952094 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.952040 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" event={"ID":"2699490ce633249cba9277b6b3cebee8","Type":"ContainerDied","Data":"3cc7a1a8a0ea50669985b78d51a93ce30caa0f4896b51f05aae5d582b8f788d6"} Jul 05 13:18:19.961473 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.961370 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qtf6g" event={"ID":"6659e427-5fd8-4cae-9675-4900eef21c94","Type":"ContainerStarted","Data":"61fb71c9bbe929faf157712f0fcc7bd92c768f70dcdf99d21e80cea2314596b4"} Jul 05 13:18:19.973026 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.972231 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-21.ec2.internal" podStartSLOduration=1.9722159179999998 podStartE2EDuration="1.972215918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:19.956518985 +0000 UTC m=+4.584884745" watchObservedRunningTime="2026-07-05 13:18:19.972215918 +0000 UTC m=+4.600581676" Jul 05 13:18:19.975490 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.975389 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" event={"ID":"175a21cc-711e-420f-8ea9-80011c45fbd5","Type":"ContainerStarted","Data":"1819350e6208b13ee926635c81cdd77fdd1f2813d531dfe3eaf13aa723951582"} Jul 05 13:18:19.993288 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.993259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" event={"ID":"77336325-afeb-4e05-8d32-01d6f60f82e7","Type":"ContainerStarted","Data":"64f2847c0461ab64e3563ffa36bb45f71f7400e8c7ea6fc601c11459e0ffe11a"} Jul 05 13:18:19.999873 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:19.999825 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sk68q" event={"ID":"373998a7-e4cd-48b0-96fa-a4962a0fbe29","Type":"ContainerStarted","Data":"c9d13c153463463240a19d84dcd21e2c47cc3d6e7ab5a9ef60adaf78942e1d26"} Jul 05 13:18:20.010662 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:20.010590 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fr86" event={"ID":"3e7cb017-f290-456f-8c62-a896ceb8e01e","Type":"ContainerStarted","Data":"3aba69c93c7bd559445864fe7d2e72022a3cc824d2a7f67edd57c7021fb34ab8"} Jul 05 13:18:20.018525 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:20.018494 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-djh4v" event={"ID":"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5","Type":"ContainerStarted","Data":"e346fab09aa2ed06b3fb5f84391efa9d444a482581d6443c041497424d828971"} Jul 05 13:18:20.032608 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:20.032576 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zpbbv" event={"ID":"3f358ef2-7b32-47c0-aaac-5c1048a34441","Type":"ContainerStarted","Data":"41d8c305ce05e1ab393ac2c899f4a21ea4eb8f0305ae01e7bed469c321a9bf72"} Jul 05 13:18:21.049916 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:21.049825 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" event={"ID":"2699490ce633249cba9277b6b3cebee8","Type":"ContainerStarted","Data":"107d36cd7072a5e42674f6ebe82948e8d6c8ef343a1c2159bb0cadcadfec504c"} Jul 05 13:18:21.419373 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:21.419335 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:21.419588 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.419497 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:21.419588 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.419564 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs podName:6cd26735-59f2-404b-981e-a44acd0f4df9 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:25.419544806 +0000 UTC m=+10.047910556 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs") pod "network-metrics-daemon-29f8w" (UID: "6cd26735-59f2-404b-981e-a44acd0f4df9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:21.520437 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:21.520398 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:21.520640 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.520611 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:21.520755 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.520678 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret podName:dcb90eb9-329e-4a35-9ced-6d86eb35eb4f nodeName:}" failed. No retries permitted until 2026-07-05 13:18:25.520659054 +0000 UTC m=+10.149024807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret") pod "global-pull-secret-syncer-h4vlk" (UID: "dcb90eb9-329e-4a35-9ced-6d86eb35eb4f") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:21.621581 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:21.620874 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:21.621581 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.621098 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:21.621581 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.621118 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:21.621581 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.621132 2579 projected.go:196] Error preparing data for projected volume kube-api-access-w9thn for pod openshift-network-diagnostics/network-check-target-6j6fs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:21.621581 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.621189 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn podName:2798e10e-cb4f-4dd5-a665-f6fc6712315c nodeName:}" failed. No retries permitted until 2026-07-05 13:18:25.62116993 +0000 UTC m=+10.249535690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9thn" (UniqueName: "kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn") pod "network-check-target-6j6fs" (UID: "2798e10e-cb4f-4dd5-a665-f6fc6712315c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:21.890301 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:21.890266 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:21.890523 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.890409 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:21.890855 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:21.890832 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:21.890960 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.890932 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:21.891036 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:21.890980 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:21.891090 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:21.891048 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:23.892460 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:23.892420 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:23.892991 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:23.892554 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:23.892991 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:23.892828 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:23.892991 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:23.892933 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:23.892991 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:23.892969 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:23.893230 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:23.893141 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:25.456304 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:25.456264 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:25.456834 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.456438 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:25.456834 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.456503 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs podName:6cd26735-59f2-404b-981e-a44acd0f4df9 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:33.456486402 +0000 UTC m=+18.084852157 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs") pod "network-metrics-daemon-29f8w" (UID: "6cd26735-59f2-404b-981e-a44acd0f4df9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:25.556996 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:25.556954 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:25.557173 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.557149 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:25.557225 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.557216 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret podName:dcb90eb9-329e-4a35-9ced-6d86eb35eb4f nodeName:}" failed. No retries permitted until 2026-07-05 13:18:33.557197395 +0000 UTC m=+18.185563145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret") pod "global-pull-secret-syncer-h4vlk" (UID: "dcb90eb9-329e-4a35-9ced-6d86eb35eb4f") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:25.658088 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:25.658047 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:25.658267 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.658255 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:25.658319 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.658275 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:25.658319 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.658289 2579 projected.go:196] Error preparing data for projected volume kube-api-access-w9thn for pod openshift-network-diagnostics/network-check-target-6j6fs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:25.658419 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.658340 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn podName:2798e10e-cb4f-4dd5-a665-f6fc6712315c nodeName:}" failed. No retries permitted until 2026-07-05 13:18:33.65832544 +0000 UTC m=+18.286691175 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9thn" (UniqueName: "kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn") pod "network-check-target-6j6fs" (UID: "2798e10e-cb4f-4dd5-a665-f6fc6712315c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:25.888452 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:25.888410 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:25.888631 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:25.888529 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:25.888631 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.888579 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:25.888631 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.888612 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:25.888852 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:25.888633 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:25.888852 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:25.888755 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:27.887975 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:27.887927 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:27.888485 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:27.888134 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:27.888609 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:27.888579 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:27.888756 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:27.888676 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:27.888841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:27.888783 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:27.888895 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:27.888865 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:29.887806 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:29.887776 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:29.888188 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:29.887921 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:29.888328 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:29.888313 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:29.888422 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:29.888404 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:29.888573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:29.888552 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:29.888712 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:29.888666 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:31.888394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:31.888355 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:31.888797 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:31.888356 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:31.888797 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:31.888468 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:31.888797 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:31.888480 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:31.888797 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:31.888580 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:31.888797 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:31.888672 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:33.522641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:33.522605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:33.523136 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.522749 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:33.523136 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.522802 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs podName:6cd26735-59f2-404b-981e-a44acd0f4df9 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:49.522787591 +0000 UTC m=+34.151153327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs") pod "network-metrics-daemon-29f8w" (UID: "6cd26735-59f2-404b-981e-a44acd0f4df9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:33.622942 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:33.622911 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:33.623091 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.623057 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:33.623141 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.623125 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret podName:dcb90eb9-329e-4a35-9ced-6d86eb35eb4f nodeName:}" failed. No retries permitted until 2026-07-05 13:18:49.623104898 +0000 UTC m=+34.251470644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret") pod "global-pull-secret-syncer-h4vlk" (UID: "dcb90eb9-329e-4a35-9ced-6d86eb35eb4f") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:33.724143 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:33.724103 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:33.724327 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.724269 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:33.724327 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.724290 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:33.724327 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.724303 2579 projected.go:196] Error preparing data for projected volume kube-api-access-w9thn for pod openshift-network-diagnostics/network-check-target-6j6fs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:33.724448 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.724354 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn podName:2798e10e-cb4f-4dd5-a665-f6fc6712315c nodeName:}" failed. No retries permitted until 2026-07-05 13:18:49.724341911 +0000 UTC m=+34.352707660 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9thn" (UniqueName: "kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn") pod "network-check-target-6j6fs" (UID: "2798e10e-cb4f-4dd5-a665-f6fc6712315c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:33.887882 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:33.887853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:33.888053 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:33.887853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:33.888053 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.887981 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:33.888053 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:33.887869 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:33.888213 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.888057 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:33.888213 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:33.888143 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:35.888260 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:35.888219 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:35.888731 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:35.888328 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:35.888731 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:35.888719 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:35.888851 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:35.888809 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:35.888908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:35.888857 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:35.888958 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:35.888903 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:37.888387 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:37.888357 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:37.889012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:37.888367 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:37.889012 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:37.888482 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:37.889012 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:37.888599 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:37.889012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:37.888651 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:37.889012 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:37.888744 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:39.088637 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.088318 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" event={"ID":"175a21cc-711e-420f-8ea9-80011c45fbd5","Type":"ContainerStarted","Data":"f2ada9d52c6b1ff46d4c047343417cd90820a58664324bde5259ea39f1cc83c7"} Jul 05 13:18:39.090118 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.090095 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" event={"ID":"77336325-afeb-4e05-8d32-01d6f60f82e7","Type":"ContainerStarted","Data":"c36033cd2444bd57870b31db9b025b4ccb921444854febf181cca6550807533c"} Jul 05 13:18:39.090221 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.090163 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" event={"ID":"77336325-afeb-4e05-8d32-01d6f60f82e7","Type":"ContainerStarted","Data":"663a5f581817f58feda4913f05432e86f07f5446e49204c26486ec82c2382b24"} Jul 05 13:18:39.091433 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.091413 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sk68q" event={"ID":"373998a7-e4cd-48b0-96fa-a4962a0fbe29","Type":"ContainerStarted","Data":"1923ce1e8738c1dc6ed779018f77b3150598ef13d117c3c0a296bc4b75fae738"} Jul 05 13:18:39.092826 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.092805 2579 generic.go:356] "Generic (PLEG): container finished" podID="3e7cb017-f290-456f-8c62-a896ceb8e01e" containerID="d57e7c9f77ca8ae306baf267fc726f6b6a98d30ad00d69965a8c5adc247bc65b" exitCode=0 Jul 05 13:18:39.092910 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.092871 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fr86" event={"ID":"3e7cb017-f290-456f-8c62-a896ceb8e01e","Type":"ContainerDied","Data":"d57e7c9f77ca8ae306baf267fc726f6b6a98d30ad00d69965a8c5adc247bc65b"} Jul 05 13:18:39.094663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.094288 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-djh4v" event={"ID":"4516d9bd-dd1b-4141-b7af-b1f2f2107ec5","Type":"ContainerStarted","Data":"5dc4f40f6bad6204dcbed2bd6d6b7b1bbc7dbbe70ed9b62e741e83a94bf79ebc"} Jul 05 13:18:39.096662 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.096641 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zpbbv" event={"ID":"3f358ef2-7b32-47c0-aaac-5c1048a34441","Type":"ContainerStarted","Data":"b91bea33096129989d84eaac7e4dc91508bfefb541f7de9ee58b315e48da4e6d"} Jul 05 13:18:39.098106 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.098084 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h9dvp" event={"ID":"cb9ca114-8751-44b3-b13a-713cf3f44f18","Type":"ContainerStarted","Data":"a57fcc234df2180cb31d62937a9638676f76dc59dd015ea0b185f9273cd522df"} Jul 05 13:18:39.099639 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.099616 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" event={"ID":"d5c57816-0ffa-4875-9f54-d56e1e4eed07","Type":"ContainerStarted","Data":"bf4e224e0954905639426b3eb168a8527f4e40357fc51f000464cac149843d35"} Jul 05 13:18:39.106345 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.106306 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sk68q" podStartSLOduration=3.557305375 podStartE2EDuration="23.106295563s" totalImagesPullingTime="19.548990188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:39.106170108 +0000 UTC m=+23.734535862" watchObservedRunningTime="2026-07-05 13:18:39.106295563 +0000 UTC m=+23.734661318" Jul 05 13:18:39.106618 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.106597 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-21.ec2.internal" podStartSLOduration=21.106591847 podStartE2EDuration="21.106591847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:21.067900041 +0000 UTC m=+5.696265802" watchObservedRunningTime="2026-07-05 13:18:39.106591847 +0000 UTC m=+23.734957601" Jul 05 13:18:39.120615 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.120584 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h9dvp" podStartSLOduration=2.579108577 podStartE2EDuration="22.120575974s" totalImagesPullingTime="19.541467397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:39.120301478 +0000 UTC m=+23.748667236" watchObservedRunningTime="2026-07-05 13:18:39.120575974 +0000 UTC m=+23.748941730" Jul 05 13:18:39.160753 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.160720 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pxxxq" podStartSLOduration=3.614290591 podStartE2EDuration="23.160707207s" totalImagesPullingTime="19.546416616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:39.160508643 +0000 UTC m=+23.788874398" watchObservedRunningTime="2026-07-05 13:18:39.160707207 +0000 UTC m=+23.789072990" Jul 05 13:18:39.177148 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.177112 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-djh4v" podStartSLOduration=3.580575137 podStartE2EDuration="23.177100704s" totalImagesPullingTime="19.596525567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:39.176982301 +0000 UTC m=+23.805348055" watchObservedRunningTime="2026-07-05 13:18:39.177100704 +0000 UTC m=+23.805466456" Jul 05 13:18:39.193653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.193623 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zpbbv" podStartSLOduration=2.653097562 podStartE2EDuration="22.193612636s" totalImagesPullingTime="19.540515074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:39.193452736 +0000 UTC m=+23.821818662" watchObservedRunningTime="2026-07-05 13:18:39.193612636 +0000 UTC m=+23.821978390" Jul 05 13:18:39.887871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.887836 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:39.887871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.887856 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:39.888139 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:39.887967 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:39.888139 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:39.887992 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:39.888139 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:39.888078 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:39.888304 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:39.888206 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:40.103462 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.103074 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qtf6g" event={"ID":"6659e427-5fd8-4cae-9675-4900eef21c94","Type":"ContainerStarted","Data":"63d094eb59618c56adcbf64a4ca1704a42591c5990a970b0b490d9f9d9e02fe5"} Jul 05 13:18:40.106458 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.106432 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" event={"ID":"77336325-afeb-4e05-8d32-01d6f60f82e7","Type":"ContainerStarted","Data":"6a39976a364d186ea2b751869d4de3439893047670c728072b8e113da7c703fa"} Jul 05 13:18:40.106583 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.106511 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" event={"ID":"77336325-afeb-4e05-8d32-01d6f60f82e7","Type":"ContainerStarted","Data":"77d50758146d41006df33f3025b37e7335349413d04039b84c7214edbff17f24"} Jul 05 13:18:40.106583 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.106561 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" event={"ID":"77336325-afeb-4e05-8d32-01d6f60f82e7","Type":"ContainerStarted","Data":"afc8ddd9c1f5f30ad50fd6af02adac255d8bd3b4cb43da2b943a6df617912121"} Jul 05 13:18:40.106709 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.106607 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" event={"ID":"77336325-afeb-4e05-8d32-01d6f60f82e7","Type":"ContainerStarted","Data":"f128723890f93401dd99d703ca3cd85058c9f6fe18dc4758e44db7dc7192e01b"} Jul 05 13:18:40.126024 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.125985 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qtf6g" podStartSLOduration=3.570570624 podStartE2EDuration="23.125972079s" totalImagesPullingTime="19.555401455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:40.125706722 +0000 UTC m=+24.754072478" watchObservedRunningTime="2026-07-05 13:18:40.125972079 +0000 UTC m=+24.754337833" Jul 05 13:18:40.316636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.316605 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 05 13:18:40.840564 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.840462 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-05T13:18:40.31663263Z","UUID":"78c5f5d8-41b5-450c-909a-e671eac44ff9","Handler":null,"Name":"","Endpoint":""} Jul 05 13:18:40.842098 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.842076 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 05 13:18:40.842201 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:40.842122 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 05 13:18:41.111850 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:41.111758 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" event={"ID":"175a21cc-711e-420f-8ea9-80011c45fbd5","Type":"ContainerStarted","Data":"32af38ad963c1bba170f382c00614a7ff3ef474b28127ed910d57b00cd6e7843"} Jul 05 13:18:41.791993 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:41.791914 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:41.792537 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:41.792514 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:41.888216 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:41.888185 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:41.888383 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:41.888312 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:41.888454 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:41.888390 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:41.888524 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:41.888498 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:41.888584 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:41.888540 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:41.888672 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:41.888643 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:42.113584 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:42.113557 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:42.114099 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:42.114078 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h9dvp" Jul 05 13:18:43.887635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:43.887460 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:43.888329 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:43.887461 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:43.888329 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:43.887733 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:43.888329 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:43.887461 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:43.888329 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:43.887814 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:43.888329 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:43.887864 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:44.119904 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:44.119865 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" event={"ID":"77336325-afeb-4e05-8d32-01d6f60f82e7","Type":"ContainerStarted","Data":"53179b9c1bcfbcf1117280d9b54863c9cd479fc71467edcc8453456b6829a5e9"} Jul 05 13:18:44.121564 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:44.121542 2579 generic.go:356] "Generic (PLEG): container finished" podID="3e7cb017-f290-456f-8c62-a896ceb8e01e" containerID="b653f4dcc1ed598997e6d278cdc926c35049a2dcdd91e4d48c881912591cf788" exitCode=0 Jul 05 13:18:44.121690 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:44.121577 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fr86" event={"ID":"3e7cb017-f290-456f-8c62-a896ceb8e01e","Type":"ContainerDied","Data":"b653f4dcc1ed598997e6d278cdc926c35049a2dcdd91e4d48c881912591cf788"} Jul 05 13:18:44.123803 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:44.123780 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" event={"ID":"175a21cc-711e-420f-8ea9-80011c45fbd5","Type":"ContainerStarted","Data":"684b8cccfa3d8ec35bbe2b89495788265be0956ef5ccae7e322c3283f94e2f9c"} Jul 05 13:18:44.161996 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:44.161923 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5bx" podStartSLOduration=5.031059356 podStartE2EDuration="27.161910764s" totalImagesPullingTime="22.130851408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:44.16166101 +0000 UTC m=+28.790026967" watchObservedRunningTime="2026-07-05 13:18:44.161910764 +0000 UTC m=+28.790276518" Jul 05 13:18:45.890442 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:45.890292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:45.890950 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:45.890292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:45.890950 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:45.890510 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:45.890950 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:45.890292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:45.890950 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:45.890581 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:45.890950 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:45.890634 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:46.133088 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:46.133058 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" event={"ID":"77336325-afeb-4e05-8d32-01d6f60f82e7","Type":"ContainerStarted","Data":"06804e02f1ceb370ca3520abf0e3d30df1173cabc32f5ccbfad48b8b47b6bd83"} Jul 05 13:18:46.133362 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:46.133325 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:46.133469 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:46.133420 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:46.133525 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:46.133484 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:46.135372 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:46.135346 2579 generic.go:356] "Generic (PLEG): container finished" podID="3e7cb017-f290-456f-8c62-a896ceb8e01e" containerID="f36608d305694ffe7d8164a9b5fcf494b1e2089d68bb70e931cab97304a60ae1" exitCode=0 Jul 05 13:18:46.135490 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:46.135391 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fr86" event={"ID":"3e7cb017-f290-456f-8c62-a896ceb8e01e","Type":"ContainerDied","Data":"f36608d305694ffe7d8164a9b5fcf494b1e2089d68bb70e931cab97304a60ae1"} Jul 05 13:18:46.149872 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:46.149821 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:46.150022 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:46.150007 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:18:46.162042 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:46.162006 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" podStartSLOduration=9.511603898 podStartE2EDuration="29.161994332s" totalImagesPullingTime="19.650390434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:46.160478725 +0000 UTC m=+30.788844479" watchObservedRunningTime="2026-07-05 13:18:46.161994332 +0000 UTC m=+30.790360124" Jul 05 13:18:47.619151 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:47.618964 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29f8w"] Jul 05 13:18:47.619576 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:47.619310 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:47.619576 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:47.619427 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:47.619661 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:47.619620 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h4vlk"] Jul 05 13:18:47.619752 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:47.619732 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:47.619865 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:47.619842 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:47.621026 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:47.621002 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6j6fs"] Jul 05 13:18:47.621119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:47.621082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:47.621185 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:47.621166 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:48.142371 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:48.142342 2579 generic.go:356] "Generic (PLEG): container finished" podID="3e7cb017-f290-456f-8c62-a896ceb8e01e" containerID="52fc9579414e44f6161595addce30589078b6bb7cf8bfdd95ad16ff52b24f365" exitCode=0 Jul 05 13:18:48.142473 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:48.142417 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fr86" event={"ID":"3e7cb017-f290-456f-8c62-a896ceb8e01e","Type":"ContainerDied","Data":"52fc9579414e44f6161595addce30589078b6bb7cf8bfdd95ad16ff52b24f365"} Jul 05 13:18:48.887910 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:48.887877 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:48.888369 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:48.887993 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:49.555430 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:49.555394 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:49.555581 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.555521 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:49.555621 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.555588 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs podName:6cd26735-59f2-404b-981e-a44acd0f4df9 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.555569907 +0000 UTC m=+66.183935654 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs") pod "network-metrics-daemon-29f8w" (UID: "6cd26735-59f2-404b-981e-a44acd0f4df9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 13:18:49.656103 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:49.656072 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:49.656256 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.656241 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:49.656319 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.656304 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret podName:dcb90eb9-329e-4a35-9ced-6d86eb35eb4f nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.656285834 +0000 UTC m=+66.284651573 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret") pod "global-pull-secret-syncer-h4vlk" (UID: "dcb90eb9-329e-4a35-9ced-6d86eb35eb4f") : object "kube-system"/"original-pull-secret" not registered Jul 05 13:18:49.756632 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:49.756602 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:49.756822 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.756794 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 13:18:49.756822 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.756815 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 13:18:49.756927 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.756827 2579 projected.go:196] Error preparing data for projected volume kube-api-access-w9thn for pod openshift-network-diagnostics/network-check-target-6j6fs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:49.756927 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.756885 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn podName:2798e10e-cb4f-4dd5-a665-f6fc6712315c nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.756867122 +0000 UTC m=+66.385232874 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9thn" (UniqueName: "kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn") pod "network-check-target-6j6fs" (UID: "2798e10e-cb4f-4dd5-a665-f6fc6712315c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 13:18:49.888449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:49.888324 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:49.888449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:49.888377 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:49.888878 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.888508 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6j6fs" podUID="2798e10e-cb4f-4dd5-a665-f6fc6712315c" Jul 05 13:18:49.888878 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:49.888568 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h4vlk" podUID="dcb90eb9-329e-4a35-9ced-6d86eb35eb4f" Jul 05 13:18:50.887656 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:50.887624 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:50.887947 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:50.887769 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29f8w" podUID="6cd26735-59f2-404b-981e-a44acd0f4df9" Jul 05 13:18:51.678785 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.678752 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-21.ec2.internal" event="NodeReady" Jul 05 13:18:51.679329 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.678900 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 05 13:18:51.722877 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.722844 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5887559dfc-7qwxt"] Jul 05 13:18:51.727911 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.727885 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.730308 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.730279 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 05 13:18:51.730441 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.730394 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 05 13:18:51.730546 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.730531 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 05 13:18:51.730966 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.730703 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ckkjd\"" Jul 05 13:18:51.735325 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.735116 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7ppkp"] Jul 05 13:18:51.736578 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.736466 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 05 13:18:51.738753 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.738734 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7ppkp" Jul 05 13:18:51.741377 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.741358 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 05 13:18:51.741377 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.741372 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 05 13:18:51.741517 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.741382 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zq4hl\"" Jul 05 13:18:51.741707 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.741672 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 05 13:18:51.748556 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.748537 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jgqxl"] Jul 05 13:18:51.752346 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.752327 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5887559dfc-7qwxt"] Jul 05 13:18:51.752445 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.752433 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.754857 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.754836 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 05 13:18:51.755065 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.755043 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7ppkp"] Jul 05 13:18:51.755484 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.755285 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t7gnk\"" Jul 05 13:18:51.755484 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.755297 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 05 13:18:51.763594 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.763572 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jgqxl"] Jul 05 13:18:51.875274 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875235 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-image-registry-private-configuration\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.875449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875288 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b0d1007-eefb-4006-8830-9f5bd22c83a3-cert\") pod \"ingress-canary-7ppkp\" (UID: \"5b0d1007-eefb-4006-8830-9f5bd22c83a3\") " pod="openshift-ingress-canary/ingress-canary-7ppkp" Jul 05 13:18:51.875449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875319 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.875449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875335 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqzw6\" (UniqueName: \"kubernetes.io/projected/60aa1aed-d099-4605-b73f-602f56a82c38-kube-api-access-kqzw6\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.875449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875352 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60aa1aed-d099-4605-b73f-602f56a82c38-tmp-dir\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.875449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875368 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dsv4\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-kube-api-access-2dsv4\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.875449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875407 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.875449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875433 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-certificates\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.875449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-bound-sa-token\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.875831 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875483 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-trusted-ca\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.875831 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875544 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-installation-pull-secrets\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.875831 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875578 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn9gs\" (UniqueName: \"kubernetes.io/projected/5b0d1007-eefb-4006-8830-9f5bd22c83a3-kube-api-access-vn9gs\") pod \"ingress-canary-7ppkp\" (UID: \"5b0d1007-eefb-4006-8830-9f5bd22c83a3\") " pod="openshift-ingress-canary/ingress-canary-7ppkp" Jul 05 13:18:51.875831 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875598 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60aa1aed-d099-4605-b73f-602f56a82c38-config-volume\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.875831 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.875617 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6c7f18b-e361-4fa5-949e-60f69bcbb880-ca-trust-extracted\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.887889 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.887430 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:18:51.887889 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.887638 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:18:51.890996 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.890891 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 13:18:51.891288 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.891267 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:18:51.891521 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.891414 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z2lc5\"" Jul 05 13:18:51.892067 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.892045 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 13:18:51.976520 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976458 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.976520 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-certificates\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.976520 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976520 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-bound-sa-token\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.976777 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976557 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-trusted-ca\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.976777 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976606 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-installation-pull-secrets\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.976777 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976643 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn9gs\" (UniqueName: \"kubernetes.io/projected/5b0d1007-eefb-4006-8830-9f5bd22c83a3-kube-api-access-vn9gs\") pod \"ingress-canary-7ppkp\" (UID: \"5b0d1007-eefb-4006-8830-9f5bd22c83a3\") " pod="openshift-ingress-canary/ingress-canary-7ppkp" Jul 05 13:18:51.976777 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976665 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60aa1aed-d099-4605-b73f-602f56a82c38-config-volume\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.976777 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976697 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6c7f18b-e361-4fa5-949e-60f69bcbb880-ca-trust-extracted\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.976777 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976728 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-image-registry-private-configuration\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.976777 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976753 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b0d1007-eefb-4006-8830-9f5bd22c83a3-cert\") pod \"ingress-canary-7ppkp\" (UID: \"5b0d1007-eefb-4006-8830-9f5bd22c83a3\") " pod="openshift-ingress-canary/ingress-canary-7ppkp" Jul 05 13:18:51.977120 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976779 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.977120 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976818 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqzw6\" (UniqueName: \"kubernetes.io/projected/60aa1aed-d099-4605-b73f-602f56a82c38-kube-api-access-kqzw6\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.977120 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976846 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60aa1aed-d099-4605-b73f-602f56a82c38-tmp-dir\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.977120 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.976873 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dsv4\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-kube-api-access-2dsv4\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.977249 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.977129 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-certificates\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.977249 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:51.976562 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:18:51.977249 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:51.977164 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5887559dfc-7qwxt: secret "image-registry-tls" not found Jul 05 13:18:51.977249 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:51.977205 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls podName:a6c7f18b-e361-4fa5-949e-60f69bcbb880 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:52.477192348 +0000 UTC m=+37.105558083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls") pod "image-registry-5887559dfc-7qwxt" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880") : secret "image-registry-tls" not found Jul 05 13:18:51.977547 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.977531 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60aa1aed-d099-4605-b73f-602f56a82c38-config-volume\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.977585 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.977530 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-trusted-ca\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.977619 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:51.977602 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:18:51.977652 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:51.977646 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls podName:60aa1aed-d099-4605-b73f-602f56a82c38 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:52.477634421 +0000 UTC m=+37.106000154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls") pod "dns-default-jgqxl" (UID: "60aa1aed-d099-4605-b73f-602f56a82c38") : secret "dns-default-metrics-tls" not found Jul 05 13:18:51.978085 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.978058 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60aa1aed-d099-4605-b73f-602f56a82c38-tmp-dir\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.978173 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.978104 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6c7f18b-e361-4fa5-949e-60f69bcbb880-ca-trust-extracted\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.981612 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.981591 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-installation-pull-secrets\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.981734 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.981653 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b0d1007-eefb-4006-8830-9f5bd22c83a3-cert\") pod \"ingress-canary-7ppkp\" (UID: \"5b0d1007-eefb-4006-8830-9f5bd22c83a3\") " pod="openshift-ingress-canary/ingress-canary-7ppkp" Jul 05 13:18:51.981734 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.981659 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-image-registry-private-configuration\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.985653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.985631 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn9gs\" (UniqueName: \"kubernetes.io/projected/5b0d1007-eefb-4006-8830-9f5bd22c83a3-kube-api-access-vn9gs\") pod \"ingress-canary-7ppkp\" (UID: \"5b0d1007-eefb-4006-8830-9f5bd22c83a3\") " pod="openshift-ingress-canary/ingress-canary-7ppkp" Jul 05 13:18:51.985947 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.985923 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqzw6\" (UniqueName: \"kubernetes.io/projected/60aa1aed-d099-4605-b73f-602f56a82c38-kube-api-access-kqzw6\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:51.986036 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.985992 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dsv4\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-kube-api-access-2dsv4\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:51.986097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:51.986080 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-bound-sa-token\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:52.048244 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:52.048212 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7ppkp" Jul 05 13:18:52.220791 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:52.220667 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7ppkp"] Jul 05 13:18:52.224127 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:18:52.224105 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0d1007_eefb_4006_8830_9f5bd22c83a3.slice/crio-cdd759e967712be6616e98d4ad77281ed85c23085171e49f5b61cc1058ff44ac WatchSource:0}: Error finding container cdd759e967712be6616e98d4ad77281ed85c23085171e49f5b61cc1058ff44ac: Status 404 returned error can't find the container with id cdd759e967712be6616e98d4ad77281ed85c23085171e49f5b61cc1058ff44ac Jul 05 13:18:52.480467 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:52.480399 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:52.480467 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:52.480439 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:52.480654 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:52.480547 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:18:52.480654 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:52.480559 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5887559dfc-7qwxt: secret "image-registry-tls" not found Jul 05 13:18:52.480654 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:52.480565 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:18:52.480654 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:52.480608 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls podName:a6c7f18b-e361-4fa5-949e-60f69bcbb880 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:53.4805913 +0000 UTC m=+38.108957033 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls") pod "image-registry-5887559dfc-7qwxt" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880") : secret "image-registry-tls" not found Jul 05 13:18:52.480654 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:52.480637 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls podName:60aa1aed-d099-4605-b73f-602f56a82c38 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:53.480615014 +0000 UTC m=+38.108980764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls") pod "dns-default-jgqxl" (UID: "60aa1aed-d099-4605-b73f-602f56a82c38") : secret "dns-default-metrics-tls" not found Jul 05 13:18:52.887974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:52.887942 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:18:52.890720 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:52.890689 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:18:52.890720 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:52.890700 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sq7qg\"" Jul 05 13:18:53.156809 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:53.156717 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7ppkp" event={"ID":"5b0d1007-eefb-4006-8830-9f5bd22c83a3","Type":"ContainerStarted","Data":"cdd759e967712be6616e98d4ad77281ed85c23085171e49f5b61cc1058ff44ac"} Jul 05 13:18:53.159837 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:53.159812 2579 generic.go:356] "Generic (PLEG): container finished" podID="3e7cb017-f290-456f-8c62-a896ceb8e01e" containerID="c3250aea4c7259dab68424539ae75fb6b109f650311e089bd28bebaf7d0f0807" exitCode=0 Jul 05 13:18:53.159962 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:53.159849 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fr86" event={"ID":"3e7cb017-f290-456f-8c62-a896ceb8e01e","Type":"ContainerDied","Data":"c3250aea4c7259dab68424539ae75fb6b109f650311e089bd28bebaf7d0f0807"} Jul 05 13:18:53.490654 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:53.490580 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:53.490654 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:53.490618 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:53.490835 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:53.490739 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:18:53.490835 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:53.490743 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:18:53.490835 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:53.490750 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5887559dfc-7qwxt: secret "image-registry-tls" not found Jul 05 13:18:53.490835 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:53.490809 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls podName:a6c7f18b-e361-4fa5-949e-60f69bcbb880 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:55.490792565 +0000 UTC m=+40.119158298 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls") pod "image-registry-5887559dfc-7qwxt" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880") : secret "image-registry-tls" not found Jul 05 13:18:53.490835 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:53.490826 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls podName:60aa1aed-d099-4605-b73f-602f56a82c38 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:55.490819659 +0000 UTC m=+40.119185392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls") pod "dns-default-jgqxl" (UID: "60aa1aed-d099-4605-b73f-602f56a82c38") : secret "dns-default-metrics-tls" not found Jul 05 13:18:54.165604 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:54.165572 2579 generic.go:356] "Generic (PLEG): container finished" podID="3e7cb017-f290-456f-8c62-a896ceb8e01e" containerID="a0db2fce980ac2cd14d50acf33f66e501670b966ae95c3a43bda966272bb583b" exitCode=0 Jul 05 13:18:54.166033 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:54.165632 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fr86" event={"ID":"3e7cb017-f290-456f-8c62-a896ceb8e01e","Type":"ContainerDied","Data":"a0db2fce980ac2cd14d50acf33f66e501670b966ae95c3a43bda966272bb583b"} Jul 05 13:18:55.171939 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:55.171713 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2fr86" event={"ID":"3e7cb017-f290-456f-8c62-a896ceb8e01e","Type":"ContainerStarted","Data":"5351940e66cf730dd4945e862ef8541d2b2bd584538aac8a5dda73a543b4f83e"} Jul 05 13:18:55.173120 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:55.173097 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7ppkp" event={"ID":"5b0d1007-eefb-4006-8830-9f5bd22c83a3","Type":"ContainerStarted","Data":"290d5b73398cd0428f361db757ace3a80eb6594a342e4577e37dd09d6e11ba92"} Jul 05 13:18:55.197043 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:55.197001 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2fr86" podStartSLOduration=8.588549703 podStartE2EDuration="38.196987409s" totalImagesPullingTime="29.608437706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:55.195538548 +0000 UTC m=+39.823904304" watchObservedRunningTime="2026-07-05 13:18:55.196987409 +0000 UTC m=+39.825353163" Jul 05 13:18:55.216993 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:55.216961 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7ppkp" podStartSLOduration=1.940580503 podStartE2EDuration="4.216948967s" totalImagesPullingTime="2.276368464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:18:55.216789901 +0000 UTC m=+39.845155656" watchObservedRunningTime="2026-07-05 13:18:55.216948967 +0000 UTC m=+39.845314714" Jul 05 13:18:55.505798 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:55.505713 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:55.505798 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:55.505751 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:55.505992 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:55.505852 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:18:55.505992 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:55.505851 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:18:55.505992 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:55.505862 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5887559dfc-7qwxt: secret "image-registry-tls" not found Jul 05 13:18:55.505992 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:55.505923 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls podName:60aa1aed-d099-4605-b73f-602f56a82c38 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:59.505906198 +0000 UTC m=+44.134271931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls") pod "dns-default-jgqxl" (UID: "60aa1aed-d099-4605-b73f-602f56a82c38") : secret "dns-default-metrics-tls" not found Jul 05 13:18:55.505992 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:55.505938 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls podName:a6c7f18b-e361-4fa5-949e-60f69bcbb880 nodeName:}" failed. No retries permitted until 2026-07-05 13:18:59.505931507 +0000 UTC m=+44.134297239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls") pod "image-registry-5887559dfc-7qwxt" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880") : secret "image-registry-tls" not found Jul 05 13:18:59.530488 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:59.530444 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:18:59.530488 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:18:59.530493 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:18:59.531007 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:59.530603 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:18:59.531007 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:59.530610 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:18:59.531007 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:59.530625 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5887559dfc-7qwxt: secret "image-registry-tls" not found Jul 05 13:18:59.531007 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:59.530674 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls podName:a6c7f18b-e361-4fa5-949e-60f69bcbb880 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:07.530656149 +0000 UTC m=+52.159021898 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls") pod "image-registry-5887559dfc-7qwxt" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880") : secret "image-registry-tls" not found Jul 05 13:18:59.531007 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:18:59.530713 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls podName:60aa1aed-d099-4605-b73f-602f56a82c38 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:07.53070451 +0000 UTC m=+52.159070246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls") pod "dns-default-jgqxl" (UID: "60aa1aed-d099-4605-b73f-602f56a82c38") : secret "dns-default-metrics-tls" not found Jul 05 13:19:07.579638 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:07.579597 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:19:07.579638 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:07.579638 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:19:07.580138 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:07.579762 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:07.580138 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:07.579777 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:07.580138 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:07.579790 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5887559dfc-7qwxt: secret "image-registry-tls" not found Jul 05 13:19:07.580138 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:07.579822 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls podName:60aa1aed-d099-4605-b73f-602f56a82c38 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:23.579807365 +0000 UTC m=+68.208173098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls") pod "dns-default-jgqxl" (UID: "60aa1aed-d099-4605-b73f-602f56a82c38") : secret "dns-default-metrics-tls" not found Jul 05 13:19:07.580138 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:07.579837 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls podName:a6c7f18b-e361-4fa5-949e-60f69bcbb880 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:23.579830994 +0000 UTC m=+68.208196727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls") pod "image-registry-5887559dfc-7qwxt" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880") : secret "image-registry-tls" not found Jul 05 13:19:16.818599 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.818565 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr"] Jul 05 13:19:16.852954 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.852926 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xbxlt"] Jul 05 13:19:16.853103 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.853035 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:16.857241 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.857221 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 05 13:19:16.862871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.862852 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:16.863145 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.863129 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 05 13:19:16.863191 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.863152 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:16.863717 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.863704 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vqtgm\"" Jul 05 13:19:16.866782 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.866764 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr"] Jul 05 13:19:16.866867 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.866790 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xbxlt"] Jul 05 13:19:16.866907 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.866868 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:16.870639 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.870625 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 05 13:19:16.870924 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.870908 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 05 13:19:16.871034 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.871015 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5pzj4\"" Jul 05 13:19:16.871446 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.871418 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 05 13:19:16.873320 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.873297 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 05 13:19:16.875107 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.875092 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 05 13:19:16.923199 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.923182 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb"] Jul 05 13:19:16.943594 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.943578 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb"] Jul 05 13:19:16.943709 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.943658 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:16.946397 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.946380 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8dwrb\"" Jul 05 13:19:16.946510 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.946383 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 05 13:19:16.946510 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.946424 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 05 13:19:16.946781 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.946761 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:19:16.948785 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.948767 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:16.948879 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.948809 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-snapshots\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:16.948879 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.948842 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-tmp\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:16.948879 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.948865 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4c58\" (UniqueName: \"kubernetes.io/projected/f9d1f33b-2c53-49f3-9697-25684d4de442-kube-api-access-v4c58\") pod \"service-ca-operator-5d46d9ffd4-jrnqr\" (UID: \"f9d1f33b-2c53-49f3-9697-25684d4de442\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:16.949037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.948929 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:16.949037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.948970 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs2rm\" (UniqueName: \"kubernetes.io/projected/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-kube-api-access-zs2rm\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:16.949037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.949014 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f9d1f33b-2c53-49f3-9697-25684d4de442-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jrnqr\" (UID: \"f9d1f33b-2c53-49f3-9697-25684d4de442\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:16.949396 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.949037 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f9d1f33b-2c53-49f3-9697-25684d4de442-config\") pod \"service-ca-operator-5d46d9ffd4-jrnqr\" (UID: \"f9d1f33b-2c53-49f3-9697-25684d4de442\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:16.949396 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:16.949117 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-serving-cert\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.049986 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.049957 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.050123 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050003 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs2rm\" (UniqueName: \"kubernetes.io/projected/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-kube-api-access-zs2rm\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.050123 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050046 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f9d1f33b-2c53-49f3-9697-25684d4de442-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jrnqr\" (UID: \"f9d1f33b-2c53-49f3-9697-25684d4de442\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:17.050123 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f9d1f33b-2c53-49f3-9697-25684d4de442-config\") pod \"service-ca-operator-5d46d9ffd4-jrnqr\" (UID: \"f9d1f33b-2c53-49f3-9697-25684d4de442\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:17.050123 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050102 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l46vr\" (UniqueName: \"kubernetes.io/projected/10a81a39-06aa-463d-95c4-9d24b424dae0-kube-api-access-l46vr\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:17.050324 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050152 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-serving-cert\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.050324 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.050324 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-snapshots\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.050324 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-tmp\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.050522 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050348 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4c58\" (UniqueName: \"kubernetes.io/projected/f9d1f33b-2c53-49f3-9697-25684d4de442-kube-api-access-v4c58\") pod \"service-ca-operator-5d46d9ffd4-jrnqr\" (UID: \"f9d1f33b-2c53-49f3-9697-25684d4de442\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:17.050522 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050377 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:17.050766 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050716 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f9d1f33b-2c53-49f3-9697-25684d4de442-config\") pod \"service-ca-operator-5d46d9ffd4-jrnqr\" (UID: \"f9d1f33b-2c53-49f3-9697-25684d4de442\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:17.050766 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050727 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.050882 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050849 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-snapshots\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.050962 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.050942 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.051020 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.051004 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-tmp\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.053910 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.053893 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f9d1f33b-2c53-49f3-9697-25684d4de442-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jrnqr\" (UID: \"f9d1f33b-2c53-49f3-9697-25684d4de442\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:17.054129 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.054111 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-serving-cert\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.070261 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.070211 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4c58\" (UniqueName: \"kubernetes.io/projected/f9d1f33b-2c53-49f3-9697-25684d4de442-kube-api-access-v4c58\") pod \"service-ca-operator-5d46d9ffd4-jrnqr\" (UID: \"f9d1f33b-2c53-49f3-9697-25684d4de442\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:17.070650 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.070633 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs2rm\" (UniqueName: \"kubernetes.io/projected/7cf49274-f63e-4e52-a228-1a1e8e1e6ac1-kube-api-access-zs2rm\") pod \"insights-operator-5c847dcfc4-xbxlt\" (UID: \"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1\") " pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.151036 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.151009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:17.151159 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:17.151141 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:17.151217 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.151166 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l46vr\" (UniqueName: \"kubernetes.io/projected/10a81a39-06aa-463d-95c4-9d24b424dae0-kube-api-access-l46vr\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:17.151217 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:17.151202 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls podName:10a81a39-06aa-463d-95c4-9d24b424dae0 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:17.651185251 +0000 UTC m=+62.279550983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-r48cb" (UID: "10a81a39-06aa-463d-95c4-9d24b424dae0") : secret "samples-operator-tls" not found Jul 05 13:19:17.162262 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.162237 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" Jul 05 13:19:17.162614 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.162596 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l46vr\" (UniqueName: \"kubernetes.io/projected/10a81a39-06aa-463d-95c4-9d24b424dae0-kube-api-access-l46vr\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:17.175006 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.174988 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" Jul 05 13:19:17.318090 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.318020 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-xbxlt"] Jul 05 13:19:17.320980 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:19:17.320913 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf49274_f63e_4e52_a228_1a1e8e1e6ac1.slice/crio-ab635e2408dba4e479fa3907871f74e81dc45846b3bb9b1d0feb8a4dc5199610 WatchSource:0}: Error finding container ab635e2408dba4e479fa3907871f74e81dc45846b3bb9b1d0feb8a4dc5199610: Status 404 returned error can't find the container with id ab635e2408dba4e479fa3907871f74e81dc45846b3bb9b1d0feb8a4dc5199610 Jul 05 13:19:17.336463 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.336398 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr"] Jul 05 13:19:17.339652 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:19:17.339630 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d1f33b_2c53_49f3_9697_25684d4de442.slice/crio-49e1b471ee32a99971753c92125b72f143c81e576c4bf57e55dd2a2c70544ca9 WatchSource:0}: Error finding container 49e1b471ee32a99971753c92125b72f143c81e576c4bf57e55dd2a2c70544ca9: Status 404 returned error can't find the container with id 49e1b471ee32a99971753c92125b72f143c81e576c4bf57e55dd2a2c70544ca9 Jul 05 13:19:17.655854 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:17.655784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:17.655993 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:17.655911 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:17.655993 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:17.655962 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls podName:10a81a39-06aa-463d-95c4-9d24b424dae0 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:18.655945954 +0000 UTC m=+63.284311689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-r48cb" (UID: "10a81a39-06aa-463d-95c4-9d24b424dae0") : secret "samples-operator-tls" not found Jul 05 13:19:18.156351 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:18.156321 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpj2z" Jul 05 13:19:18.228338 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:18.228302 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" event={"ID":"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1","Type":"ContainerStarted","Data":"ab635e2408dba4e479fa3907871f74e81dc45846b3bb9b1d0feb8a4dc5199610"} Jul 05 13:19:18.229645 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:18.229612 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" event={"ID":"f9d1f33b-2c53-49f3-9697-25684d4de442","Type":"ContainerStarted","Data":"49e1b471ee32a99971753c92125b72f143c81e576c4bf57e55dd2a2c70544ca9"} Jul 05 13:19:18.664659 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:18.664623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:18.664856 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:18.664810 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:18.664911 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:18.664866 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls podName:10a81a39-06aa-463d-95c4-9d24b424dae0 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:20.664850722 +0000 UTC m=+65.293216455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-r48cb" (UID: "10a81a39-06aa-463d-95c4-9d24b424dae0") : secret "samples-operator-tls" not found Jul 05 13:19:20.237483 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:20.237449 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" event={"ID":"f9d1f33b-2c53-49f3-9697-25684d4de442","Type":"ContainerStarted","Data":"cd35a13cf9889e493932f1867ce22233f9870f9fe21fa77f971e739f80b74e38"} Jul 05 13:19:20.239021 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:20.238993 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" event={"ID":"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1","Type":"ContainerStarted","Data":"abad18f971cea03753068ee836346e11f67a2b9ea3b5af613c84920d587973d5"} Jul 05 13:19:20.294300 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:20.294249 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" podStartSLOduration=2.131844434 podStartE2EDuration="4.294215556s" totalImagesPullingTime="2.162371122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:20.293493488 +0000 UTC m=+64.921859243" watchObservedRunningTime="2026-07-05 13:19:20.294215556 +0000 UTC m=+64.922581312" Jul 05 13:19:20.294562 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:20.294538 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" podStartSLOduration=2.148262557 podStartE2EDuration="4.29453172s" totalImagesPullingTime="2.146269163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:20.264229527 +0000 UTC m=+64.892595282" watchObservedRunningTime="2026-07-05 13:19:20.29453172 +0000 UTC m=+64.922897476" Jul 05 13:19:20.681918 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:20.681879 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:20.682103 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:20.682026 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:20.682103 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:20.682091 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls podName:10a81a39-06aa-463d-95c4-9d24b424dae0 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:24.682076981 +0000 UTC m=+69.310442713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-r48cb" (UID: "10a81a39-06aa-463d-95c4-9d24b424dae0") : secret "samples-operator-tls" not found Jul 05 13:19:21.283778 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.283745 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2cqxt"] Jul 05 13:19:21.286664 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.286012 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.289672 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.289478 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 05 13:19:21.290941 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.290736 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 05 13:19:21.291765 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.291747 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n6z8z\"" Jul 05 13:19:21.370637 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.370609 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2cqxt"] Jul 05 13:19:21.386816 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.386791 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.386943 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.386822 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6zg9\" (UniqueName: \"kubernetes.io/projected/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-api-access-s6zg9\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.386943 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.386873 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42b6e001-bc2d-48a0-b007-2a1154128b43-data-volume\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.386943 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.386935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.387053 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.387022 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42b6e001-bc2d-48a0-b007-2a1154128b43-crio-socket\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.487564 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.487532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42b6e001-bc2d-48a0-b007-2a1154128b43-data-volume\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.487564 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.487565 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.487741 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.487619 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42b6e001-bc2d-48a0-b007-2a1154128b43-crio-socket\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.487741 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.487672 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.487741 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.487722 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6zg9\" (UniqueName: \"kubernetes.io/projected/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-api-access-s6zg9\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.487888 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.487763 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:21.487888 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.487778 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42b6e001-bc2d-48a0-b007-2a1154128b43-crio-socket\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.487888 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.487835 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:21.487888 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.487842 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.98782326 +0000 UTC m=+66.616189019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : configmap "kube-rbac-proxy" not found Jul 05 13:19:21.488074 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.487871 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42b6e001-bc2d-48a0-b007-2a1154128b43-data-volume\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.488401 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.487901 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:21.987890783 +0000 UTC m=+66.616256527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:21.503787 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.503757 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6zg9\" (UniqueName: \"kubernetes.io/projected/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-api-access-s6zg9\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.588344 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.588314 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:19:21.607741 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.607719 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 13:19:21.608722 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.608710 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 13:19:21.608780 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.608764 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs podName:6cd26735-59f2-404b-981e-a44acd0f4df9 nodeName:}" failed. No retries permitted until 2026-07-05 13:20:25.608750255 +0000 UTC m=+130.237115988 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs") pod "network-metrics-daemon-29f8w" (UID: "6cd26735-59f2-404b-981e-a44acd0f4df9") : secret "metrics-daemon-secret" not found Jul 05 13:19:21.689238 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.689210 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:19:21.692369 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.692355 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 13:19:21.701952 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.701921 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb90eb9-329e-4a35-9ced-6d86eb35eb4f-original-pull-secret\") pod \"global-pull-secret-syncer-h4vlk\" (UID: \"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f\") " pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:19:21.789953 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.789918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:19:21.793256 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.793239 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 13:19:21.802913 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.802892 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 13:19:21.813046 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.813029 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9thn\" (UniqueName: \"kubernetes.io/projected/2798e10e-cb4f-4dd5-a665-f6fc6712315c-kube-api-access-w9thn\") pod \"network-check-target-6j6fs\" (UID: \"2798e10e-cb4f-4dd5-a665-f6fc6712315c\") " pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:19:21.903187 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.903134 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z2lc5\"" Jul 05 13:19:21.906738 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.906723 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h4vlk" Jul 05 13:19:21.911447 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.911430 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:19:21.991127 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.991099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.991223 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:21.991190 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:21.991912 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.991322 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:21.991912 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.991388 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:22.991367941 +0000 UTC m=+67.619733693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : configmap "kube-rbac-proxy" not found Jul 05 13:19:21.991912 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.991847 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:21.991912 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:21.991907 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:22.991891738 +0000 UTC m=+67.620257473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:22.040026 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:22.039998 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h4vlk"] Jul 05 13:19:22.042840 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:19:22.042818 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb90eb9_329e_4a35_9ced_6d86eb35eb4f.slice/crio-69c59988c2b2d0467eb330e751f70b692c156a1b499a8e72c222f795fd2ff351 WatchSource:0}: Error finding container 69c59988c2b2d0467eb330e751f70b692c156a1b499a8e72c222f795fd2ff351: Status 404 returned error can't find the container with id 69c59988c2b2d0467eb330e751f70b692c156a1b499a8e72c222f795fd2ff351 Jul 05 13:19:22.056868 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:22.056846 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6j6fs"] Jul 05 13:19:22.057902 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:19:22.057880 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2798e10e_cb4f_4dd5_a665_f6fc6712315c.slice/crio-d51ddad76864600f31519d6bb7322afd9b31ac6d85fc2041ca7dad277f2b5959 WatchSource:0}: Error finding container d51ddad76864600f31519d6bb7322afd9b31ac6d85fc2041ca7dad277f2b5959: Status 404 returned error can't find the container with id d51ddad76864600f31519d6bb7322afd9b31ac6d85fc2041ca7dad277f2b5959 Jul 05 13:19:22.246941 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:22.246864 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6j6fs" event={"ID":"2798e10e-cb4f-4dd5-a665-f6fc6712315c","Type":"ContainerStarted","Data":"d51ddad76864600f31519d6bb7322afd9b31ac6d85fc2041ca7dad277f2b5959"} Jul 05 13:19:22.248052 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:22.248017 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h4vlk" event={"ID":"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f","Type":"ContainerStarted","Data":"69c59988c2b2d0467eb330e751f70b692c156a1b499a8e72c222f795fd2ff351"} Jul 05 13:19:23.000481 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:23.000223 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:23.000481 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:23.000322 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:23.000481 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:23.000400 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:23.000481 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:23.000422 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:23.000481 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:23.000471 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:25.0004507 +0000 UTC m=+69.628816478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:23.000481 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:23.000493 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:25.000484202 +0000 UTC m=+69.628849939 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : configmap "kube-rbac-proxy" not found Jul 05 13:19:23.475260 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:23.475229 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zpbbv_3f358ef2-7b32-47c0-aaac-5c1048a34441/dns-node-resolver/0.log" Jul 05 13:19:23.605646 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:23.605610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:19:23.605841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:23.605654 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:19:23.605841 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:23.605832 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 05 13:19:23.605975 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:23.605847 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5887559dfc-7qwxt: secret "image-registry-tls" not found Jul 05 13:19:23.605975 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:23.605904 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls podName:a6c7f18b-e361-4fa5-949e-60f69bcbb880 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:55.605885712 +0000 UTC m=+100.234251460 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls") pod "image-registry-5887559dfc-7qwxt" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880") : secret "image-registry-tls" not found Jul 05 13:19:23.606395 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:23.606297 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 13:19:23.606395 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:23.606358 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls podName:60aa1aed-d099-4605-b73f-602f56a82c38 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:55.606341695 +0000 UTC m=+100.234707440 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls") pod "dns-default-jgqxl" (UID: "60aa1aed-d099-4605-b73f-602f56a82c38") : secret "dns-default-metrics-tls" not found Jul 05 13:19:24.469959 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:24.469922 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sk68q_373998a7-e4cd-48b0-96fa-a4962a0fbe29/node-ca/0.log" Jul 05 13:19:24.714400 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:24.714365 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:24.714594 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:24.714524 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 05 13:19:24.714594 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:24.714586 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls podName:10a81a39-06aa-463d-95c4-9d24b424dae0 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:32.714572807 +0000 UTC m=+77.342938540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-r48cb" (UID: "10a81a39-06aa-463d-95c4-9d24b424dae0") : secret "samples-operator-tls" not found Jul 05 13:19:25.016417 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:25.016382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:25.016569 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:25.016441 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:25.016569 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:25.016526 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:25.016569 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:25.016538 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:25.016667 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:25.016593 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:29.016575828 +0000 UTC m=+73.644941561 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:25.016667 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:25.016612 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:29.01660194 +0000 UTC m=+73.644967672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : configmap "kube-rbac-proxy" not found Jul 05 13:19:25.083850 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:25.083816 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7ppkp_5b0d1007-eefb-4006-8830-9f5bd22c83a3/serve-healthcheck-canary/0.log" Jul 05 13:19:27.264026 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:27.263977 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6j6fs" event={"ID":"2798e10e-cb4f-4dd5-a665-f6fc6712315c","Type":"ContainerStarted","Data":"c3cc033fb5a65bcf8888edc7c169571510d40feadd946c5f7238358a052c94eb"} Jul 05 13:19:27.264489 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:27.264076 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:19:27.265272 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:27.265250 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h4vlk" event={"ID":"dcb90eb9-329e-4a35-9ced-6d86eb35eb4f","Type":"ContainerStarted","Data":"6d1dd52e775f63032e66ca623bdc56836125d034edacabff4134e8f8a41d290f"} Jul 05 13:19:27.306584 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:27.306539 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6j6fs" podStartSLOduration=65.856036505 podStartE2EDuration="1m10.306525913s" totalImagesPullingTime="4.450489408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:27.305703215 +0000 UTC m=+71.934068968" watchObservedRunningTime="2026-07-05 13:19:27.306525913 +0000 UTC m=+71.934891668" Jul 05 13:19:27.334065 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:27.334020 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h4vlk" podStartSLOduration=65.864272421 podStartE2EDuration="1m10.334006194s" totalImagesPullingTime="4.469733773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:27.333494733 +0000 UTC m=+71.961860488" watchObservedRunningTime="2026-07-05 13:19:27.334006194 +0000 UTC m=+71.962371948" Jul 05 13:19:29.049623 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:29.049589 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:29.050098 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:29.049668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:29.050098 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:29.049741 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 13:19:29.050098 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:29.049778 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:29.050098 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:29.049818 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:37.049801827 +0000 UTC m=+81.678167562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : configmap "kube-rbac-proxy" not found Jul 05 13:19:29.050098 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:29.049835 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:37.049827434 +0000 UTC m=+81.678193166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:32.776363 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:32.776331 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:32.778915 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:32.778890 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10a81a39-06aa-463d-95c4-9d24b424dae0-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-r48cb\" (UID: \"10a81a39-06aa-463d-95c4-9d24b424dae0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:32.852052 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:32.852024 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" Jul 05 13:19:33.075891 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:33.075855 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb"] Jul 05 13:19:33.282289 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:33.282259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" event={"ID":"10a81a39-06aa-463d-95c4-9d24b424dae0","Type":"ContainerStarted","Data":"efd8ebf9ac1269aeb91c54258c101ec9159037de144a90192bf371bed8d5534e"} Jul 05 13:19:35.290211 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:35.290172 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" event={"ID":"10a81a39-06aa-463d-95c4-9d24b424dae0","Type":"ContainerStarted","Data":"5b1c9863aeb67a7ba2810ad301d27f812298de074cdcede96f0d4306aeac6d84"} Jul 05 13:19:35.290573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:35.290223 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" event={"ID":"10a81a39-06aa-463d-95c4-9d24b424dae0","Type":"ContainerStarted","Data":"19ee9416cb612be2f7db84c6ff9d54762e030a61416c4df4adac96db57d5298f"} Jul 05 13:19:35.331659 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:35.331612 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-r48cb" podStartSLOduration=17.778458097 podStartE2EDuration="19.331597359s" totalImagesPullingTime="1.553139262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:35.331367788 +0000 UTC m=+79.959733546" watchObservedRunningTime="2026-07-05 13:19:35.331597359 +0000 UTC m=+79.959963113" Jul 05 13:19:37.111430 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:37.111321 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:37.111430 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:37.111399 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:37.111863 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:37.111488 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 13:19:37.111863 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:19:37.111544 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls podName:42b6e001-bc2d-48a0-b007-2a1154128b43 nodeName:}" failed. No retries permitted until 2026-07-05 13:19:53.11152848 +0000 UTC m=+97.739894216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cqxt" (UID: "42b6e001-bc2d-48a0-b007-2a1154128b43") : secret "insights-runtime-extractor-tls" not found Jul 05 13:19:37.111936 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:37.111865 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42b6e001-bc2d-48a0-b007-2a1154128b43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:53.132914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.132868 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:53.135318 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.135296 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42b6e001-bc2d-48a0-b007-2a1154128b43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cqxt\" (UID: \"42b6e001-bc2d-48a0-b007-2a1154128b43\") " pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:53.390096 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.390027 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z"] Jul 05 13:19:53.392534 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.392516 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7"] Jul 05 13:19:53.392648 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.392630 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" Jul 05 13:19:53.394410 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.394389 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" Jul 05 13:19:53.395809 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.395788 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 05 13:19:53.396576 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.396560 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 05 13:19:53.396576 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.396569 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-j2b9j\"" Jul 05 13:19:53.397132 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.397118 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2cqxt" Jul 05 13:19:53.406827 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.406788 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 05 13:19:53.406919 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.406896 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 05 13:19:53.407063 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.407047 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 05 13:19:53.408004 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.407768 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-t9hpx\"" Jul 05 13:19:53.408004 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.407805 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 05 13:19:53.423333 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.423310 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z"] Jul 05 13:19:53.460965 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.460903 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7"] Jul 05 13:19:53.529055 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.528942 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-689745f7bc-m5626"] Jul 05 13:19:53.531660 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.531643 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.536496 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.536473 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ccebd28f-1a0e-4f00-9447-c005b8122899-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2m2z\" (UID: \"ccebd28f-1a0e-4f00-9447-c005b8122899\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" Jul 05 13:19:53.536623 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.536514 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqjst\" (UniqueName: \"kubernetes.io/projected/487f7687-e4b1-4eb8-ade8-1b234a90dd8e-kube-api-access-kqjst\") pod \"managed-serviceaccount-addon-agent-598595cb94-wflp7\" (UID: \"487f7687-e4b1-4eb8-ade8-1b234a90dd8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" Jul 05 13:19:53.536623 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.536579 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/487f7687-e4b1-4eb8-ade8-1b234a90dd8e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-598595cb94-wflp7\" (UID: \"487f7687-e4b1-4eb8-ade8-1b234a90dd8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" Jul 05 13:19:53.536749 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.536706 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ccebd28f-1a0e-4f00-9447-c005b8122899-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-z2m2z\" (UID: \"ccebd28f-1a0e-4f00-9447-c005b8122899\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" Jul 05 13:19:53.543918 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.543893 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 05 13:19:53.545044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.545029 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mn2hn\"" Jul 05 13:19:53.545112 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.545068 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 05 13:19:53.551298 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.551279 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 05 13:19:53.551367 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.551347 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 05 13:19:53.551367 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.551356 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 05 13:19:53.551474 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.551359 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 05 13:19:53.551474 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.551408 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 05 13:19:53.616335 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.616305 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2cqxt"] Jul 05 13:19:53.620105 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:19:53.620075 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b6e001_bc2d_48a0_b007_2a1154128b43.slice/crio-7fe265eded287643e480d2a703e1f4296a32e23333a9a578b3d925334b3e0a4e WatchSource:0}: Error finding container 7fe265eded287643e480d2a703e1f4296a32e23333a9a578b3d925334b3e0a4e: Status 404 returned error can't find the container with id 7fe265eded287643e480d2a703e1f4296a32e23333a9a578b3d925334b3e0a4e Jul 05 13:19:53.623439 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.623418 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689745f7bc-m5626"] Jul 05 13:19:53.637955 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.637931 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-serving-cert\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.638059 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.637961 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-oauth-config\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.638059 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.638033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ccebd28f-1a0e-4f00-9447-c005b8122899-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-z2m2z\" (UID: \"ccebd28f-1a0e-4f00-9447-c005b8122899\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" Jul 05 13:19:53.638168 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.638090 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ccebd28f-1a0e-4f00-9447-c005b8122899-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2m2z\" (UID: \"ccebd28f-1a0e-4f00-9447-c005b8122899\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" Jul 05 13:19:53.638168 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.638127 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dhlk\" (UniqueName: \"kubernetes.io/projected/41cad192-9110-4a08-b9e2-a629c982f5ff-kube-api-access-4dhlk\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.638168 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.638155 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-oauth-serving-cert\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.638319 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.638211 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjst\" (UniqueName: \"kubernetes.io/projected/487f7687-e4b1-4eb8-ade8-1b234a90dd8e-kube-api-access-kqjst\") pod \"managed-serviceaccount-addon-agent-598595cb94-wflp7\" (UID: \"487f7687-e4b1-4eb8-ade8-1b234a90dd8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" Jul 05 13:19:53.638319 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.638236 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-console-config\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.638319 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.638312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/487f7687-e4b1-4eb8-ade8-1b234a90dd8e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-598595cb94-wflp7\" (UID: \"487f7687-e4b1-4eb8-ade8-1b234a90dd8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" Jul 05 13:19:53.638503 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.638344 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-service-ca\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.638890 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.638846 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ccebd28f-1a0e-4f00-9447-c005b8122899-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-z2m2z\" (UID: \"ccebd28f-1a0e-4f00-9447-c005b8122899\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" Jul 05 13:19:53.640817 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.640767 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ccebd28f-1a0e-4f00-9447-c005b8122899-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-z2m2z\" (UID: \"ccebd28f-1a0e-4f00-9447-c005b8122899\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" Jul 05 13:19:53.640911 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.640824 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/487f7687-e4b1-4eb8-ade8-1b234a90dd8e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-598595cb94-wflp7\" (UID: \"487f7687-e4b1-4eb8-ade8-1b234a90dd8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" Jul 05 13:19:53.699722 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.699671 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqjst\" (UniqueName: \"kubernetes.io/projected/487f7687-e4b1-4eb8-ade8-1b234a90dd8e-kube-api-access-kqjst\") pod \"managed-serviceaccount-addon-agent-598595cb94-wflp7\" (UID: \"487f7687-e4b1-4eb8-ade8-1b234a90dd8e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" Jul 05 13:19:53.703621 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.703602 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" Jul 05 13:19:53.728859 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.728838 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" Jul 05 13:19:53.738961 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.738936 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dhlk\" (UniqueName: \"kubernetes.io/projected/41cad192-9110-4a08-b9e2-a629c982f5ff-kube-api-access-4dhlk\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.739066 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.738965 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-oauth-serving-cert\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.739066 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.738987 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-console-config\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.739066 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.739047 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-service-ca\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.739230 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.739103 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-serving-cert\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.739230 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.739127 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-oauth-config\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.739754 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.739728 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-oauth-serving-cert\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.740024 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.739740 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-console-config\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.740024 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.739840 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-service-ca\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.741718 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.741697 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-oauth-config\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.742080 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.742059 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-serving-cert\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.776988 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.776955 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dhlk\" (UniqueName: \"kubernetes.io/projected/41cad192-9110-4a08-b9e2-a629c982f5ff-kube-api-access-4dhlk\") pod \"console-689745f7bc-m5626\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.835863 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.835836 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85d4fbbb6b-7rn5l"] Jul 05 13:19:53.839024 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.838993 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:53.840169 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.840018 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:19:53.872044 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.872015 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d4fbbb6b-7rn5l"] Jul 05 13:19:53.884778 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.884755 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 05 13:19:53.927201 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:19:53.927174 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod487f7687_e4b1_4eb8_ade8_1b234a90dd8e.slice/crio-216b7812c71eece0f9eeecf07a19b1986ca9fee5a48e3fd1da285df194264407 WatchSource:0}: Error finding container 216b7812c71eece0f9eeecf07a19b1986ca9fee5a48e3fd1da285df194264407: Status 404 returned error can't find the container with id 216b7812c71eece0f9eeecf07a19b1986ca9fee5a48e3fd1da285df194264407 Jul 05 13:19:53.927842 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.927774 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z"] Jul 05 13:19:53.928981 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.928965 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7"] Jul 05 13:19:53.934543 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:19:53.934516 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccebd28f_1a0e_4f00_9447_c005b8122899.slice/crio-75219e0fc79a40bdfefec695b875339455caadd3f9b980a2239e5c341bff2765 WatchSource:0}: Error finding container 75219e0fc79a40bdfefec695b875339455caadd3f9b980a2239e5c341bff2765: Status 404 returned error can't find the container with id 75219e0fc79a40bdfefec695b875339455caadd3f9b980a2239e5c341bff2765 Jul 05 13:19:53.943082 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.943060 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-service-ca\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:53.943185 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.943126 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-oauth-serving-cert\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:53.943261 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.943241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-config\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:53.943314 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.943278 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7c65\" (UniqueName: \"kubernetes.io/projected/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-kube-api-access-z7c65\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:53.943314 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.943298 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-serving-cert\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:53.943388 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.943314 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-oauth-config\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:53.943427 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:53.943391 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-trusted-ca-bundle\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.044195 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.044163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-config\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.044318 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.044212 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7c65\" (UniqueName: \"kubernetes.io/projected/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-kube-api-access-z7c65\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.044382 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.044346 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-serving-cert\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.044452 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.044385 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-oauth-config\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.044452 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.044427 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-trusted-ca-bundle\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.044565 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.044480 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-service-ca\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.044565 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.044509 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-oauth-serving-cert\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.045016 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.044918 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-config\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.045308 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.045282 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-trusted-ca-bundle\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.045383 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.045292 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-oauth-serving-cert\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.045383 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.045340 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-service-ca\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.047301 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.047271 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-serving-cert\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.047390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.047373 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-oauth-config\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.125611 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.125584 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7c65\" (UniqueName: \"kubernetes.io/projected/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-kube-api-access-z7c65\") pod \"console-85d4fbbb6b-7rn5l\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.151211 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.151182 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:19:54.304329 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.304261 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689745f7bc-m5626"] Jul 05 13:19:54.310593 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:19:54.310562 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41cad192_9110_4a08_b9e2_a629c982f5ff.slice/crio-2d8acc50f2964021bb79d6106e1745f90f7047c1d0364bb57b54c19ce0602817 WatchSource:0}: Error finding container 2d8acc50f2964021bb79d6106e1745f90f7047c1d0364bb57b54c19ce0602817: Status 404 returned error can't find the container with id 2d8acc50f2964021bb79d6106e1745f90f7047c1d0364bb57b54c19ce0602817 Jul 05 13:19:54.351570 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.351541 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689745f7bc-m5626" event={"ID":"41cad192-9110-4a08-b9e2-a629c982f5ff","Type":"ContainerStarted","Data":"2d8acc50f2964021bb79d6106e1745f90f7047c1d0364bb57b54c19ce0602817"} Jul 05 13:19:54.353421 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.353312 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" event={"ID":"487f7687-e4b1-4eb8-ade8-1b234a90dd8e","Type":"ContainerStarted","Data":"216b7812c71eece0f9eeecf07a19b1986ca9fee5a48e3fd1da285df194264407"} Jul 05 13:19:54.356551 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.356528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" event={"ID":"ccebd28f-1a0e-4f00-9447-c005b8122899","Type":"ContainerStarted","Data":"75219e0fc79a40bdfefec695b875339455caadd3f9b980a2239e5c341bff2765"} Jul 05 13:19:54.356706 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.356649 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d4fbbb6b-7rn5l"] Jul 05 13:19:54.358549 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:54.358517 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cqxt" event={"ID":"42b6e001-bc2d-48a0-b007-2a1154128b43","Type":"ContainerStarted","Data":"7fe265eded287643e480d2a703e1f4296a32e23333a9a578b3d925334b3e0a4e"} Jul 05 13:19:54.361560 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:19:54.361535 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e97403f_909a_4d0f_bce4_ad9ee3e342bc.slice/crio-d8ba5874015ce8ee869ef1537d451350724b8a93fb56f27d2c6da5f28547c846 WatchSource:0}: Error finding container d8ba5874015ce8ee869ef1537d451350724b8a93fb56f27d2c6da5f28547c846: Status 404 returned error can't find the container with id d8ba5874015ce8ee869ef1537d451350724b8a93fb56f27d2c6da5f28547c846 Jul 05 13:19:55.365624 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:55.365531 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d4fbbb6b-7rn5l" event={"ID":"5e97403f-909a-4d0f-bce4-ad9ee3e342bc","Type":"ContainerStarted","Data":"d8ba5874015ce8ee869ef1537d451350724b8a93fb56f27d2c6da5f28547c846"} Jul 05 13:19:55.661107 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:55.660501 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:19:55.661107 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:55.660551 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:19:55.667289 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:55.663727 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60aa1aed-d099-4605-b73f-602f56a82c38-metrics-tls\") pod \"dns-default-jgqxl\" (UID: \"60aa1aed-d099-4605-b73f-602f56a82c38\") " pod="openshift-dns/dns-default-jgqxl" Jul 05 13:19:55.679467 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:55.678968 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"image-registry-5887559dfc-7qwxt\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:19:55.941988 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:55.941912 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ckkjd\"" Jul 05 13:19:55.949875 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:55.949850 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:19:55.965134 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:55.965089 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t7gnk\"" Jul 05 13:19:55.973322 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:55.973299 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jgqxl" Jul 05 13:19:56.372028 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:56.371984 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" event={"ID":"ccebd28f-1a0e-4f00-9447-c005b8122899","Type":"ContainerStarted","Data":"714cd9b4d6a109ca3bd00bedb9d3a7ec02097ddeb5a2e90927800831797a748a"} Jul 05 13:19:56.390658 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:56.390597 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-z2m2z" podStartSLOduration=1.8318619969999999 podStartE2EDuration="3.39057542s" totalImagesPullingTime="1.558713423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:19:56.390548131 +0000 UTC m=+101.018913886" watchObservedRunningTime="2026-07-05 13:19:56.39057542 +0000 UTC m=+101.018941177" Jul 05 13:19:58.270792 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:19:58.270761 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6j6fs" Jul 05 13:20:01.131090 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:01.131057 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jgqxl"] Jul 05 13:20:01.468738 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:20:01.468672 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60aa1aed_d099_4605_b73f_602f56a82c38.slice/crio-1adaefb4965b40fe341afb76922ffda0e150d6be66d54815163335b65bfbb59f WatchSource:0}: Error finding container 1adaefb4965b40fe341afb76922ffda0e150d6be66d54815163335b65bfbb59f: Status 404 returned error can't find the container with id 1adaefb4965b40fe341afb76922ffda0e150d6be66d54815163335b65bfbb59f Jul 05 13:20:01.686822 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:01.686782 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5887559dfc-7qwxt"] Jul 05 13:20:01.689951 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:20:01.689920 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6c7f18b_e361_4fa5_949e_60f69bcbb880.slice/crio-9fe151ff822e7c8d6a0bb26c8868d7ab9348fb3c7dd0f2f009a7e6fd4a53f11d WatchSource:0}: Error finding container 9fe151ff822e7c8d6a0bb26c8868d7ab9348fb3c7dd0f2f009a7e6fd4a53f11d: Status 404 returned error can't find the container with id 9fe151ff822e7c8d6a0bb26c8868d7ab9348fb3c7dd0f2f009a7e6fd4a53f11d Jul 05 13:20:02.395351 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.395080 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689745f7bc-m5626" event={"ID":"41cad192-9110-4a08-b9e2-a629c982f5ff","Type":"ContainerStarted","Data":"68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69"} Jul 05 13:20:02.396746 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.396708 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" event={"ID":"487f7687-e4b1-4eb8-ade8-1b234a90dd8e","Type":"ContainerStarted","Data":"072e54bc4d1aba4a50c1a2a7386414a0240ef4ec09d1230db9ddfeed61a1b1f8"} Jul 05 13:20:02.398771 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.398746 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" event={"ID":"a6c7f18b-e361-4fa5-949e-60f69bcbb880","Type":"ContainerStarted","Data":"8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748"} Jul 05 13:20:02.398873 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.398794 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" event={"ID":"a6c7f18b-e361-4fa5-949e-60f69bcbb880","Type":"ContainerStarted","Data":"9fe151ff822e7c8d6a0bb26c8868d7ab9348fb3c7dd0f2f009a7e6fd4a53f11d"} Jul 05 13:20:02.398873 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.398819 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:20:02.400216 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.400187 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jgqxl" event={"ID":"60aa1aed-d099-4605-b73f-602f56a82c38","Type":"ContainerStarted","Data":"1adaefb4965b40fe341afb76922ffda0e150d6be66d54815163335b65bfbb59f"} Jul 05 13:20:02.401844 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.401819 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cqxt" event={"ID":"42b6e001-bc2d-48a0-b007-2a1154128b43","Type":"ContainerStarted","Data":"6299920a066ce1badc51d1855e637f63138df57e3df71fc19f69a1194b9aeb53"} Jul 05 13:20:02.403348 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.403324 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d4fbbb6b-7rn5l" event={"ID":"5e97403f-909a-4d0f-bce4-ad9ee3e342bc","Type":"ContainerStarted","Data":"fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43"} Jul 05 13:20:02.432485 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.432437 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-689745f7bc-m5626" podStartSLOduration=2.74143563 podStartE2EDuration="9.432419719s" totalImagesPullingTime="6.690984089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:02.431825661 +0000 UTC m=+107.060191416" watchObservedRunningTime="2026-07-05 13:20:02.432419719 +0000 UTC m=+107.060785475" Jul 05 13:20:02.503755 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.503708 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" podStartSLOduration=105.503692336 podStartE2EDuration="1m45.503692336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:02.502998984 +0000 UTC m=+107.131364739" watchObservedRunningTime="2026-07-05 13:20:02.503692336 +0000 UTC m=+107.132058088" Jul 05 13:20:02.566025 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.565969 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-598595cb94-wflp7" podStartSLOduration=2.525811417 podStartE2EDuration="9.5659482s" totalImagesPullingTime="7.040136783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:02.535488727 +0000 UTC m=+107.163854482" watchObservedRunningTime="2026-07-05 13:20:02.5659482 +0000 UTC m=+107.194313956" Jul 05 13:20:02.566197 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:02.566076 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85d4fbbb6b-7rn5l" podStartSLOduration=2.423025233 podStartE2EDuration="9.566068015s" totalImagesPullingTime="7.143042782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:02.565057027 +0000 UTC m=+107.193422784" watchObservedRunningTime="2026-07-05 13:20:02.566068015 +0000 UTC m=+107.194433774" Jul 05 13:20:03.410698 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:03.410645 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cqxt" event={"ID":"42b6e001-bc2d-48a0-b007-2a1154128b43","Type":"ContainerStarted","Data":"9fcbb38f73f1bb2e9481bb40414036a9f3ee1cf0371d38a72d703cff54da59b7"} Jul 05 13:20:03.842064 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:03.842021 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:20:03.842242 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:03.842100 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:20:03.847367 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:03.847344 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:20:04.151555 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.151508 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:20:04.151757 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.151592 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:20:04.157762 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.157733 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:20:04.416739 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.416601 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jgqxl" event={"ID":"60aa1aed-d099-4605-b73f-602f56a82c38","Type":"ContainerStarted","Data":"b65893302589a592507cd6f9294ba2f3e448ca48c77a7faa7ad4a24ba4357689"} Jul 05 13:20:04.417329 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.417133 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jgqxl" Jul 05 13:20:04.417329 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.417171 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jgqxl" event={"ID":"60aa1aed-d099-4605-b73f-602f56a82c38","Type":"ContainerStarted","Data":"314daa30ae021499bf690b189f5a1f64c3b2480226d0c0db2d1f96dca3b10aee"} Jul 05 13:20:04.420949 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.420926 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:20:04.422125 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.422105 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:20:04.501587 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.501119 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jgqxl" podStartSLOduration=71.508380721 podStartE2EDuration="1m13.501097946s" totalImagesPullingTime="1.992717225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:04.500664868 +0000 UTC m=+109.129030625" watchObservedRunningTime="2026-07-05 13:20:04.501097946 +0000 UTC m=+109.129463698" Jul 05 13:20:04.797534 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:04.797460 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689745f7bc-m5626"] Jul 05 13:20:05.421924 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:05.421889 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cqxt" event={"ID":"42b6e001-bc2d-48a0-b007-2a1154128b43","Type":"ContainerStarted","Data":"71ba47c24d4b4c5ef9439db1789511e30aad5bc3230754232e27ced897e875b5"} Jul 05 13:20:05.466743 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:05.466699 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2cqxt" podStartSLOduration=33.261049818 podStartE2EDuration="44.466670154s" totalImagesPullingTime="11.205620336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 13:19:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:05.465869955 +0000 UTC m=+110.094235710" watchObservedRunningTime="2026-07-05 13:20:05.466670154 +0000 UTC m=+110.095035909" Jul 05 13:20:15.427599 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:15.427562 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jgqxl" Jul 05 13:20:16.021791 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.021757 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2"] Jul 05 13:20:16.027651 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.027629 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.032801 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.032782 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 05 13:20:16.033214 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.033194 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 05 13:20:16.033298 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.033214 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 05 13:20:16.033730 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.033574 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-csb7z\"" Jul 05 13:20:16.033730 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.033574 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 05 13:20:16.033730 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.033615 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 05 13:20:16.077113 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.077092 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2"] Jul 05 13:20:16.087000 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.086980 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6"] Jul 05 13:20:16.090353 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.090338 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.094331 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.094310 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wwbck\"" Jul 05 13:20:16.094431 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.094364 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 05 13:20:16.094646 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.094633 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 05 13:20:16.097830 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.097815 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 05 13:20:16.116372 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116352 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa7e5237-b83e-47d9-b627-bbd790997257-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.116466 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116384 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.116466 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116405 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa7e5237-b83e-47d9-b627-bbd790997257-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.116466 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116425 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt2th\" (UniqueName: \"kubernetes.io/projected/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-api-access-qt2th\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.116604 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116502 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.116604 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa7e5237-b83e-47d9-b627-bbd790997257-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.116604 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116578 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.116604 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116598 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jwk5\" (UniqueName: \"kubernetes.io/projected/aa7e5237-b83e-47d9-b627-bbd790997257-kube-api-access-6jwk5\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.116751 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116649 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.116751 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.116699 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.120178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.120159 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jml4c"] Jul 05 13:20:16.123887 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.123868 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6"] Jul 05 13:20:16.123971 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.123951 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.127528 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.127513 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 05 13:20:16.127710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.127695 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8xvgv\"" Jul 05 13:20:16.127775 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.127758 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 05 13:20:16.128032 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.128017 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 05 13:20:16.217521 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.217499 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.217653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.217537 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-textfile\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.217653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.217559 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa7e5237-b83e-47d9-b627-bbd790997257-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.217653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.217631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.217786 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.217660 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp2gn\" (UniqueName: \"kubernetes.io/projected/dc45372b-1060-4080-a52b-de78d72d2055-kube-api-access-rp2gn\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.218102 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218077 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jwk5\" (UniqueName: \"kubernetes.io/projected/aa7e5237-b83e-47d9-b627-bbd790997257-kube-api-access-6jwk5\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.218190 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218131 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc45372b-1060-4080-a52b-de78d72d2055-sys\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.218257 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218184 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.218257 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218226 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc45372b-1060-4080-a52b-de78d72d2055-root\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.218363 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.218363 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218304 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-tls\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.218363 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218339 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc45372b-1060-4080-a52b-de78d72d2055-metrics-client-ca\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.220311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218870 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-accelerators-collector-config\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.220311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218951 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa7e5237-b83e-47d9-b627-bbd790997257-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.220311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.218991 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.220311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.219023 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa7e5237-b83e-47d9-b627-bbd790997257-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.220311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.219065 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt2th\" (UniqueName: \"kubernetes.io/projected/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-api-access-qt2th\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.220311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.219351 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-wtmp\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.220311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.219400 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.220311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.219557 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.220813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.220703 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.220926 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.220706 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa7e5237-b83e-47d9-b627-bbd790997257-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.221164 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.221131 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa7e5237-b83e-47d9-b627-bbd790997257-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.221253 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.221209 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.221317 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.221301 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.221545 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.221522 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.223443 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.222879 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa7e5237-b83e-47d9-b627-bbd790997257-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.238027 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.237995 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jwk5\" (UniqueName: \"kubernetes.io/projected/aa7e5237-b83e-47d9-b627-bbd790997257-kube-api-access-6jwk5\") pod \"openshift-state-metrics-54cf96dfcf-sjnf2\" (UID: \"aa7e5237-b83e-47d9-b627-bbd790997257\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.241872 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.241851 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt2th\" (UniqueName: \"kubernetes.io/projected/c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb-kube-api-access-qt2th\") pod \"kube-state-metrics-7c9dbb8b7d-p95c6\" (UID: \"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.320359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320333 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc45372b-1060-4080-a52b-de78d72d2055-metrics-client-ca\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320490 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-accelerators-collector-config\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320490 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-wtmp\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320490 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320490 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-textfile\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320490 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320487 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp2gn\" (UniqueName: \"kubernetes.io/projected/dc45372b-1060-4080-a52b-de78d72d2055-kube-api-access-rp2gn\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320780 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320513 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc45372b-1060-4080-a52b-de78d72d2055-sys\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320780 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320543 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-wtmp\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320780 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320564 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc45372b-1060-4080-a52b-de78d72d2055-sys\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320780 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc45372b-1060-4080-a52b-de78d72d2055-root\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320780 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-tls\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.320780 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320750 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc45372b-1060-4080-a52b-de78d72d2055-root\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.321093 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320851 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-textfile\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.321093 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.320991 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc45372b-1060-4080-a52b-de78d72d2055-metrics-client-ca\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.321093 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.321075 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-accelerators-collector-config\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.322702 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.322664 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.322905 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.322887 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc45372b-1060-4080-a52b-de78d72d2055-node-exporter-tls\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.333261 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.333240 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp2gn\" (UniqueName: \"kubernetes.io/projected/dc45372b-1060-4080-a52b-de78d72d2055-kube-api-access-rp2gn\") pod \"node-exporter-jml4c\" (UID: \"dc45372b-1060-4080-a52b-de78d72d2055\") " pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.336119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.336105 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" Jul 05 13:20:16.398849 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.398816 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" Jul 05 13:20:16.432308 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.432276 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jml4c" Jul 05 13:20:16.447618 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:20:16.447578 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc45372b_1060_4080_a52b_de78d72d2055.slice/crio-bd2048e20456c26b3cde56c9ca53b7794cfcc05b86c571d00732110a77047a05 WatchSource:0}: Error finding container bd2048e20456c26b3cde56c9ca53b7794cfcc05b86c571d00732110a77047a05: Status 404 returned error can't find the container with id bd2048e20456c26b3cde56c9ca53b7794cfcc05b86c571d00732110a77047a05 Jul 05 13:20:16.458231 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.458118 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jml4c" event={"ID":"dc45372b-1060-4080-a52b-de78d72d2055","Type":"ContainerStarted","Data":"bd2048e20456c26b3cde56c9ca53b7794cfcc05b86c571d00732110a77047a05"} Jul 05 13:20:16.526496 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.526471 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2"] Jul 05 13:20:16.529420 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:20:16.529399 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa7e5237_b83e_47d9_b627_bbd790997257.slice/crio-b3540851b19168169851bf092e45fae91e756930a5da191029c73dfb94e95e7b WatchSource:0}: Error finding container b3540851b19168169851bf092e45fae91e756930a5da191029c73dfb94e95e7b: Status 404 returned error can't find the container with id b3540851b19168169851bf092e45fae91e756930a5da191029c73dfb94e95e7b Jul 05 13:20:16.595290 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.595235 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6"] Jul 05 13:20:16.597371 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:20:16.597348 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5087350_cb82_4fb3_8bdb_4ea64a0f0bfb.slice/crio-25f245ff7aaba1573cd75db2c8add8c66063558433b473c0bec5131a9435c627 WatchSource:0}: Error finding container 25f245ff7aaba1573cd75db2c8add8c66063558433b473c0bec5131a9435c627: Status 404 returned error can't find the container with id 25f245ff7aaba1573cd75db2c8add8c66063558433b473c0bec5131a9435c627 Jul 05 13:20:16.854476 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.854402 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5887559dfc-7qwxt"] Jul 05 13:20:16.860057 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:16.860030 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:20:17.463374 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:17.463334 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" event={"ID":"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb","Type":"ContainerStarted","Data":"25f245ff7aaba1573cd75db2c8add8c66063558433b473c0bec5131a9435c627"} Jul 05 13:20:17.465669 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:17.465633 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" event={"ID":"aa7e5237-b83e-47d9-b627-bbd790997257","Type":"ContainerStarted","Data":"67d54dbc5aa4ce37d81e34a3e68ffd693802567f468f5a57c751412ca38ac7e7"} Jul 05 13:20:17.465805 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:17.465721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" event={"ID":"aa7e5237-b83e-47d9-b627-bbd790997257","Type":"ContainerStarted","Data":"68724d6b2868dbd01855c92c1eadca98876ccf05bd68b97cecf7f94c94823f07"} Jul 05 13:20:17.465805 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:17.465753 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" event={"ID":"aa7e5237-b83e-47d9-b627-bbd790997257","Type":"ContainerStarted","Data":"b3540851b19168169851bf092e45fae91e756930a5da191029c73dfb94e95e7b"} Jul 05 13:20:18.471857 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:18.471761 2579 generic.go:356] "Generic (PLEG): container finished" podID="dc45372b-1060-4080-a52b-de78d72d2055" containerID="f7fb9384ba6cd740f2897b17df65fad5b3e57b83b662f0e16677e54c16b5e968" exitCode=0 Jul 05 13:20:18.472309 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:18.471851 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jml4c" event={"ID":"dc45372b-1060-4080-a52b-de78d72d2055","Type":"ContainerDied","Data":"f7fb9384ba6cd740f2897b17df65fad5b3e57b83b662f0e16677e54c16b5e968"} Jul 05 13:20:18.474083 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:18.474051 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" event={"ID":"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb","Type":"ContainerStarted","Data":"e06bb78030d79a391bf00cbe8f58f971f22a136805ce882eec2d16bb326bc920"} Jul 05 13:20:18.474200 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:18.474119 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" event={"ID":"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb","Type":"ContainerStarted","Data":"4db1e7b155a5f57e01f8c5d168d72756015c76ec12b89c6ccd7faae38d639ed1"} Jul 05 13:20:18.474200 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:18.474148 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" event={"ID":"c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb","Type":"ContainerStarted","Data":"b5a2c900d70489d5fb784f57a7b021f4db82dd70263e2505dbad275925e9336d"} Jul 05 13:20:18.476177 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:18.476151 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" event={"ID":"aa7e5237-b83e-47d9-b627-bbd790997257","Type":"ContainerStarted","Data":"06dadfcad153cda70a6127e49103aff4738c333ae8e6f3f4b8d5b819ec1ba0e5"} Jul 05 13:20:18.555235 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:18.555178 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-sjnf2" podStartSLOduration=2.297124314 podStartE2EDuration="3.555163423s" totalImagesPullingTime="1.258039109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:18.55391964 +0000 UTC m=+123.182285395" watchObservedRunningTime="2026-07-05 13:20:18.555163423 +0000 UTC m=+123.183529177" Jul 05 13:20:18.600271 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:18.595963 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-p95c6" podStartSLOduration=1.289413743 podStartE2EDuration="2.595942665s" totalImagesPullingTime="1.306528922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:18.594860196 +0000 UTC m=+123.223225951" watchObservedRunningTime="2026-07-05 13:20:18.595942665 +0000 UTC m=+123.224308432" Jul 05 13:20:19.482056 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:19.482005 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jml4c" event={"ID":"dc45372b-1060-4080-a52b-de78d72d2055","Type":"ContainerStarted","Data":"9323693593f9c4f4881ce175e11c1d3c316698cabb189e352b5afd99654fd9d8"} Jul 05 13:20:19.482477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:19.482109 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jml4c" event={"ID":"dc45372b-1060-4080-a52b-de78d72d2055","Type":"ContainerStarted","Data":"54eddf5f9a0600bb47269968d754892e123d9a06a0d4a421252948a6064fdfbf"} Jul 05 13:20:19.526251 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:19.526207 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jml4c" podStartSLOduration=1.978832385 podStartE2EDuration="3.526188197s" totalImagesPullingTime="1.455370675s" totalInitContainerRuntime="91.985137ms" isStatefulPod=true podCreationTimestamp="2026-07-05 13:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:19.525224044 +0000 UTC m=+124.153589792" watchObservedRunningTime="2026-07-05 13:20:19.526188197 +0000 UTC m=+124.154553952" Jul 05 13:20:20.598324 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.598297 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65fcdddf4b-x65dd"] Jul 05 13:20:20.606960 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.606936 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.617836 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.617816 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65fcdddf4b-x65dd"] Jul 05 13:20:20.659433 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.659404 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-trusted-ca-bundle\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.659539 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.659442 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-config\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.659539 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.659460 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-serving-cert\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.659539 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.659481 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-oauth-config\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.659539 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.659526 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-service-ca\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.659726 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.659557 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4rht\" (UniqueName: \"kubernetes.io/projected/2c053694-e213-4fd7-b60e-e9170ee60a5a-kube-api-access-l4rht\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.659726 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.659597 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-oauth-serving-cert\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.759977 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.759955 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-trusted-ca-bundle\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.760123 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.759990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-config\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.760123 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.760010 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-serving-cert\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.760123 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.760026 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-oauth-config\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.760236 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.760220 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-service-ca\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.760280 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.760258 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4rht\" (UniqueName: \"kubernetes.io/projected/2c053694-e213-4fd7-b60e-e9170ee60a5a-kube-api-access-l4rht\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.760323 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.760309 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-oauth-serving-cert\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.760883 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.760856 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-config\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.761011 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.760922 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-service-ca\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.761011 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.760959 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-trusted-ca-bundle\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.761209 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.761188 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-oauth-serving-cert\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.762575 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.762548 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-serving-cert\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.762669 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.762576 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-oauth-config\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.769614 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.769595 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4rht\" (UniqueName: \"kubernetes.io/projected/2c053694-e213-4fd7-b60e-e9170ee60a5a-kube-api-access-l4rht\") pod \"console-65fcdddf4b-x65dd\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:20.916374 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:20.916292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:21.057799 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:21.057708 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65fcdddf4b-x65dd"] Jul 05 13:20:21.065428 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:20:21.065401 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c053694_e213_4fd7_b60e_e9170ee60a5a.slice/crio-420a337001024f0add5cbca9691c81bce7b209d4dcca9143fb97d779998f7d5d WatchSource:0}: Error finding container 420a337001024f0add5cbca9691c81bce7b209d4dcca9143fb97d779998f7d5d: Status 404 returned error can't find the container with id 420a337001024f0add5cbca9691c81bce7b209d4dcca9143fb97d779998f7d5d Jul 05 13:20:21.490670 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:21.490632 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65fcdddf4b-x65dd" event={"ID":"2c053694-e213-4fd7-b60e-e9170ee60a5a","Type":"ContainerStarted","Data":"c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c"} Jul 05 13:20:21.490854 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:21.490693 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65fcdddf4b-x65dd" event={"ID":"2c053694-e213-4fd7-b60e-e9170ee60a5a","Type":"ContainerStarted","Data":"420a337001024f0add5cbca9691c81bce7b209d4dcca9143fb97d779998f7d5d"} Jul 05 13:20:21.510877 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:21.510831 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65fcdddf4b-x65dd" podStartSLOduration=1.510816976 podStartE2EDuration="1.510816976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:21.509652925 +0000 UTC m=+126.138018681" watchObservedRunningTime="2026-07-05 13:20:21.510816976 +0000 UTC m=+126.139182777" Jul 05 13:20:22.332663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.332628 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:20:22.336331 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.336311 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.340061 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.340028 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 05 13:20:22.340748 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.340724 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 05 13:20:22.340865 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.340798 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-t7jfj\"" Jul 05 13:20:22.341178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.341154 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 05 13:20:22.341434 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.341413 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 05 13:20:22.341491 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.341450 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 05 13:20:22.341580 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.341558 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 05 13:20:22.341636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.341609 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 05 13:20:22.341793 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.341772 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 05 13:20:22.341861 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.341815 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 05 13:20:22.341961 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.341938 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 05 13:20:22.342115 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.342038 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 05 13:20:22.342115 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.342072 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-br2ukfr548p9j\"" Jul 05 13:20:22.342259 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.342226 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 05 13:20:22.342560 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.342543 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 05 13:20:22.342669 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.342544 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 05 13:20:22.345889 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.345874 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 05 13:20:22.360686 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.360660 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:20:22.374120 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374100 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374120 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374126 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374296 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374143 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-web-config\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374296 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374296 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374220 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374296 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374244 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374296 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374272 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374296 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374295 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374323 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374387 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-config\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374409 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374426 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374443 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374561 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374577 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-config-out\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374590 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzk4z\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-kube-api-access-jzk4z\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.374601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.374608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475349 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475326 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475461 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475461 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475461 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475398 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-config-out\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475591 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzk4z\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-kube-api-access-jzk4z\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475591 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475541 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475591 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475576 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475783 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475609 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475783 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475636 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-web-config\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475783 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475660 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475783 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475783 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475771 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.475783 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475776 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.476076 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.476076 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475824 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.476076 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475872 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.476076 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.476076 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475941 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-config\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.476076 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475963 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.476076 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.475992 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.476076 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.476025 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.478357 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.476768 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.478357 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.476895 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.478357 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.476939 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.478357 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.477614 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.478633 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.478610 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.478768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.478744 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.480623 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.479725 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.480623 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.480009 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-web-config\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.480623 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.480199 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.480623 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.480285 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.480929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.480903 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.481367 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.481322 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.481531 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.481507 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-config-out\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.481608 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.481560 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.481667 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.481651 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-config\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.481807 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.481787 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.482476 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.482460 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.482958 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.482943 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.498728 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.498703 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzk4z\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-kube-api-access-jzk4z\") pod \"prometheus-k8s-0\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.650301 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.650218 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:22.795013 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:22.794985 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:20:22.798665 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:20:22.798643 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2afb2873_412d_4248_8502_bd783a3fc381.slice/crio-c7d24b9b9d24bfd5669ff04def0eaf5cffa480ac18e877809b1a8ffc578be41b WatchSource:0}: Error finding container c7d24b9b9d24bfd5669ff04def0eaf5cffa480ac18e877809b1a8ffc578be41b: Status 404 returned error can't find the container with id c7d24b9b9d24bfd5669ff04def0eaf5cffa480ac18e877809b1a8ffc578be41b Jul 05 13:20:23.498654 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:23.498624 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerStarted","Data":"c7d24b9b9d24bfd5669ff04def0eaf5cffa480ac18e877809b1a8ffc578be41b"} Jul 05 13:20:24.503498 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:24.503464 2579 generic.go:356] "Generic (PLEG): container finished" podID="2afb2873-412d-4248-8502-bd783a3fc381" containerID="4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075" exitCode=0 Jul 05 13:20:24.503859 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:24.503540 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerDied","Data":"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075"} Jul 05 13:20:25.705848 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:25.705810 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:20:25.708465 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:25.708439 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cd26735-59f2-404b-981e-a44acd0f4df9-metrics-certs\") pod \"network-metrics-daemon-29f8w\" (UID: \"6cd26735-59f2-404b-981e-a44acd0f4df9\") " pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:20:25.900156 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:25.900125 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sq7qg\"" Jul 05 13:20:25.908421 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:25.908393 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29f8w" Jul 05 13:20:26.065781 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:26.065756 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29f8w"] Jul 05 13:20:26.068669 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:20:26.068644 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd26735_59f2_404b_981e_a44acd0f4df9.slice/crio-731f646f800ab558bdcc11cbc2416ddfc64e648ea5822b28df39459385a7bdee WatchSource:0}: Error finding container 731f646f800ab558bdcc11cbc2416ddfc64e648ea5822b28df39459385a7bdee: Status 404 returned error can't find the container with id 731f646f800ab558bdcc11cbc2416ddfc64e648ea5822b28df39459385a7bdee Jul 05 13:20:26.512766 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:26.512732 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29f8w" event={"ID":"6cd26735-59f2-404b-981e-a44acd0f4df9","Type":"ContainerStarted","Data":"731f646f800ab558bdcc11cbc2416ddfc64e648ea5822b28df39459385a7bdee"} Jul 05 13:20:28.523586 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:28.523494 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerStarted","Data":"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb"} Jul 05 13:20:28.524045 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:28.523621 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerStarted","Data":"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc"} Jul 05 13:20:28.525593 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:28.525556 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29f8w" event={"ID":"6cd26735-59f2-404b-981e-a44acd0f4df9","Type":"ContainerStarted","Data":"dce248f348b521abf369f971cff9c37982bd4877074a55877fbe63048e485ff0"} Jul 05 13:20:28.525736 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:28.525629 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29f8w" event={"ID":"6cd26735-59f2-404b-981e-a44acd0f4df9","Type":"ContainerStarted","Data":"560ecf57397863739a55dafdcb913a48b41e4c667a10f139048478968d5a2315"} Jul 05 13:20:28.543905 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:28.543852 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-29f8w" podStartSLOduration=130.539774153 podStartE2EDuration="2m12.543832168s" totalImagesPullingTime="2.004058015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:28.543398097 +0000 UTC m=+133.171763852" watchObservedRunningTime="2026-07-05 13:20:28.543832168 +0000 UTC m=+133.172197925" Jul 05 13:20:30.536847 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:30.536814 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerStarted","Data":"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d"} Jul 05 13:20:30.537196 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:30.536892 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerStarted","Data":"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940"} Jul 05 13:20:30.537196 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:30.536969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerStarted","Data":"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48"} Jul 05 13:20:30.537196 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:30.537008 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerStarted","Data":"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5"} Jul 05 13:20:30.566072 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:30.566012 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.446636437 podStartE2EDuration="8.565984381s" totalImagesPullingTime="6.100832614s" totalInitContainerRuntime="18.51533ms" isStatefulPod=false podCreationTimestamp="2026-07-05 13:20:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:20:30.56396369 +0000 UTC m=+135.192329456" watchObservedRunningTime="2026-07-05 13:20:30.565984381 +0000 UTC m=+135.194350137" Jul 05 13:20:30.917325 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:30.917294 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:30.917507 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:30.917350 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:30.922404 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:30.922382 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:31.445359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.445318 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-689745f7bc-m5626" podUID="41cad192-9110-4a08-b9e2-a629c982f5ff" containerName="console" containerID="cri-o://68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69" gracePeriod=15 Jul 05 13:20:31.544601 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.544579 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:20:31.610037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.610013 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d4fbbb6b-7rn5l"] Jul 05 13:20:31.793294 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.793268 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:20:31.858535 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.858511 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-service-ca\") pod \"41cad192-9110-4a08-b9e2-a629c982f5ff\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " Jul 05 13:20:31.858714 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.858558 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dhlk\" (UniqueName: \"kubernetes.io/projected/41cad192-9110-4a08-b9e2-a629c982f5ff-kube-api-access-4dhlk\") pod \"41cad192-9110-4a08-b9e2-a629c982f5ff\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " Jul 05 13:20:31.858714 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.858594 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-oauth-serving-cert\") pod \"41cad192-9110-4a08-b9e2-a629c982f5ff\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " Jul 05 13:20:31.858714 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.858644 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-console-config\") pod \"41cad192-9110-4a08-b9e2-a629c982f5ff\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " Jul 05 13:20:31.858858 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.858749 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-oauth-config\") pod \"41cad192-9110-4a08-b9e2-a629c982f5ff\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " Jul 05 13:20:31.858858 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.858785 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-serving-cert\") pod \"41cad192-9110-4a08-b9e2-a629c982f5ff\" (UID: \"41cad192-9110-4a08-b9e2-a629c982f5ff\") " Jul 05 13:20:31.858966 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.858933 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-service-ca" (OuterVolumeSpecName: "service-ca") pod "41cad192-9110-4a08-b9e2-a629c982f5ff" (UID: "41cad192-9110-4a08-b9e2-a629c982f5ff"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:31.859022 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.859003 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "41cad192-9110-4a08-b9e2-a629c982f5ff" (UID: "41cad192-9110-4a08-b9e2-a629c982f5ff"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:31.859077 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.859065 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-service-ca\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:31.859130 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.859082 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-oauth-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:31.859429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.859407 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-console-config" (OuterVolumeSpecName: "console-config") pod "41cad192-9110-4a08-b9e2-a629c982f5ff" (UID: "41cad192-9110-4a08-b9e2-a629c982f5ff"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:31.861067 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.861044 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "41cad192-9110-4a08-b9e2-a629c982f5ff" (UID: "41cad192-9110-4a08-b9e2-a629c982f5ff"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:31.861416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.861398 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "41cad192-9110-4a08-b9e2-a629c982f5ff" (UID: "41cad192-9110-4a08-b9e2-a629c982f5ff"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:31.861460 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.861430 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41cad192-9110-4a08-b9e2-a629c982f5ff-kube-api-access-4dhlk" (OuterVolumeSpecName: "kube-api-access-4dhlk") pod "41cad192-9110-4a08-b9e2-a629c982f5ff" (UID: "41cad192-9110-4a08-b9e2-a629c982f5ff"). InnerVolumeSpecName "kube-api-access-4dhlk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:31.960324 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.960300 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dhlk\" (UniqueName: \"kubernetes.io/projected/41cad192-9110-4a08-b9e2-a629c982f5ff-kube-api-access-4dhlk\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:31.960324 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.960322 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41cad192-9110-4a08-b9e2-a629c982f5ff-console-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:31.960494 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.960332 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-oauth-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:31.960494 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:31.960340 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41cad192-9110-4a08-b9e2-a629c982f5ff-console-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:32.545425 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.545391 2579 generic.go:356] "Generic (PLEG): container finished" podID="41cad192-9110-4a08-b9e2-a629c982f5ff" containerID="68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69" exitCode=0 Jul 05 13:20:32.545908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.545440 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689745f7bc-m5626" Jul 05 13:20:32.545908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.545483 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689745f7bc-m5626" event={"ID":"41cad192-9110-4a08-b9e2-a629c982f5ff","Type":"ContainerDied","Data":"68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69"} Jul 05 13:20:32.545908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.545533 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689745f7bc-m5626" event={"ID":"41cad192-9110-4a08-b9e2-a629c982f5ff","Type":"ContainerDied","Data":"2d8acc50f2964021bb79d6106e1745f90f7047c1d0364bb57b54c19ce0602817"} Jul 05 13:20:32.545908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.545554 2579 scope.go:117] "RemoveContainer" containerID="68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69" Jul 05 13:20:32.554512 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.554497 2579 scope.go:117] "RemoveContainer" containerID="68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69" Jul 05 13:20:32.554827 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:20:32.554805 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69\": container with ID starting with 68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69 not found: ID does not exist" containerID="68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69" Jul 05 13:20:32.554900 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.554835 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69"} err="failed to get container status \"68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69\": rpc error: code = NotFound desc = could not find container \"68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69\": container with ID starting with 68532b9d023ce84cd26d2653e6983de4be6905ff8e451492e3b6e59b7be37e69 not found: ID does not exist" Jul 05 13:20:32.562051 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.562029 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689745f7bc-m5626"] Jul 05 13:20:32.566333 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.566314 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-689745f7bc-m5626"] Jul 05 13:20:32.651342 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:32.651318 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:20:33.892454 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:33.892418 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41cad192-9110-4a08-b9e2-a629c982f5ff" path="/var/lib/kubelet/pods/41cad192-9110-4a08-b9e2-a629c982f5ff/volumes" Jul 05 13:20:40.576265 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:40.576229 2579 generic.go:356] "Generic (PLEG): container finished" podID="f9d1f33b-2c53-49f3-9697-25684d4de442" containerID="cd35a13cf9889e493932f1867ce22233f9870f9fe21fa77f971e739f80b74e38" exitCode=0 Jul 05 13:20:40.576647 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:40.576296 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" event={"ID":"f9d1f33b-2c53-49f3-9697-25684d4de442","Type":"ContainerDied","Data":"cd35a13cf9889e493932f1867ce22233f9870f9fe21fa77f971e739f80b74e38"} Jul 05 13:20:40.576647 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:40.576591 2579 scope.go:117] "RemoveContainer" containerID="cd35a13cf9889e493932f1867ce22233f9870f9fe21fa77f971e739f80b74e38" Jul 05 13:20:41.581692 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:41.581644 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jrnqr" event={"ID":"f9d1f33b-2c53-49f3-9697-25684d4de442","Type":"ContainerStarted","Data":"2a78babe4f52e90bcc9ee070d2d28e3f91dfb7319744b23c9e38da6794a1ce95"} Jul 05 13:20:41.879171 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:41.879092 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" podUID="a6c7f18b-e361-4fa5-949e-60f69bcbb880" containerName="registry" containerID="cri-o://8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748" gracePeriod=30 Jul 05 13:20:42.139459 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.139409 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:20:42.240888 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.240862 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6c7f18b-e361-4fa5-949e-60f69bcbb880-ca-trust-extracted\") pod \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " Jul 05 13:20:42.241019 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.240895 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-certificates\") pod \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " Jul 05 13:20:42.241019 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.240921 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dsv4\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-kube-api-access-2dsv4\") pod \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " Jul 05 13:20:42.241019 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.240939 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-installation-pull-secrets\") pod \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " Jul 05 13:20:42.241019 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.240979 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") pod \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " Jul 05 13:20:42.241220 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.241036 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-image-registry-private-configuration\") pod \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " Jul 05 13:20:42.241220 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.241073 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-trusted-ca\") pod \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " Jul 05 13:20:42.241220 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.241114 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-bound-sa-token\") pod \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\" (UID: \"a6c7f18b-e361-4fa5-949e-60f69bcbb880\") " Jul 05 13:20:42.242359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.242329 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a6c7f18b-e361-4fa5-949e-60f69bcbb880" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:42.244358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.242412 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a6c7f18b-e361-4fa5-949e-60f69bcbb880" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:42.249361 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.249325 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-kube-api-access-2dsv4" (OuterVolumeSpecName: "kube-api-access-2dsv4") pod "a6c7f18b-e361-4fa5-949e-60f69bcbb880" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880"). InnerVolumeSpecName "kube-api-access-2dsv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:42.249361 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.249342 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a6c7f18b-e361-4fa5-949e-60f69bcbb880" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:42.249541 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.249421 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a6c7f18b-e361-4fa5-949e-60f69bcbb880" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:42.249541 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.249422 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a6c7f18b-e361-4fa5-949e-60f69bcbb880" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:42.249755 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.249722 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a6c7f18b-e361-4fa5-949e-60f69bcbb880" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:42.256320 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.256295 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c7f18b-e361-4fa5-949e-60f69bcbb880-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a6c7f18b-e361-4fa5-949e-60f69bcbb880" (UID: "a6c7f18b-e361-4fa5-949e-60f69bcbb880"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:20:42.342951 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.342915 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-trusted-ca\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:42.342951 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.342948 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-bound-sa-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:42.342951 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.342957 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6c7f18b-e361-4fa5-949e-60f69bcbb880-ca-trust-extracted\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:42.343116 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.342967 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-certificates\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:42.343116 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.342976 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dsv4\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-kube-api-access-2dsv4\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:42.343116 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.342984 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-installation-pull-secrets\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:42.343116 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.342992 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6c7f18b-e361-4fa5-949e-60f69bcbb880-registry-tls\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:42.343116 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.343000 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6c7f18b-e361-4fa5-949e-60f69bcbb880-image-registry-private-configuration\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:42.586671 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.586639 2579 generic.go:356] "Generic (PLEG): container finished" podID="a6c7f18b-e361-4fa5-949e-60f69bcbb880" containerID="8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748" exitCode=0 Jul 05 13:20:42.587053 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.586715 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" event={"ID":"a6c7f18b-e361-4fa5-949e-60f69bcbb880","Type":"ContainerDied","Data":"8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748"} Jul 05 13:20:42.587053 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.586730 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" Jul 05 13:20:42.587053 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.586789 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5887559dfc-7qwxt" event={"ID":"a6c7f18b-e361-4fa5-949e-60f69bcbb880","Type":"ContainerDied","Data":"9fe151ff822e7c8d6a0bb26c8868d7ab9348fb3c7dd0f2f009a7e6fd4a53f11d"} Jul 05 13:20:42.587053 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.586811 2579 scope.go:117] "RemoveContainer" containerID="8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748" Jul 05 13:20:42.595243 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.595228 2579 scope.go:117] "RemoveContainer" containerID="8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748" Jul 05 13:20:42.595499 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:20:42.595480 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748\": container with ID starting with 8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748 not found: ID does not exist" containerID="8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748" Jul 05 13:20:42.595574 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.595506 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748"} err="failed to get container status \"8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748\": rpc error: code = NotFound desc = could not find container \"8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748\": container with ID starting with 8c85070cc9b4dea6a84a8378b37a76ca230dd7b42e39103dfb959dc5a699c748 not found: ID does not exist" Jul 05 13:20:42.607196 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.607176 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5887559dfc-7qwxt"] Jul 05 13:20:42.612332 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:42.612313 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5887559dfc-7qwxt"] Jul 05 13:20:43.891963 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:43.891935 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6c7f18b-e361-4fa5-949e-60f69bcbb880" path="/var/lib/kubelet/pods/a6c7f18b-e361-4fa5-949e-60f69bcbb880/volumes" Jul 05 13:20:50.618769 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:50.618738 2579 generic.go:356] "Generic (PLEG): container finished" podID="7cf49274-f63e-4e52-a228-1a1e8e1e6ac1" containerID="abad18f971cea03753068ee836346e11f67a2b9ea3b5af613c84920d587973d5" exitCode=0 Jul 05 13:20:50.619135 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:50.618822 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" event={"ID":"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1","Type":"ContainerDied","Data":"abad18f971cea03753068ee836346e11f67a2b9ea3b5af613c84920d587973d5"} Jul 05 13:20:50.619135 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:50.619111 2579 scope.go:117] "RemoveContainer" containerID="abad18f971cea03753068ee836346e11f67a2b9ea3b5af613c84920d587973d5" Jul 05 13:20:51.624088 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:51.624061 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-xbxlt" event={"ID":"7cf49274-f63e-4e52-a228-1a1e8e1e6ac1","Type":"ContainerStarted","Data":"6a0deb29cddb382a20faafc60d496f1f146cb654cb8bfd1d40204a138e6abc53"} Jul 05 13:20:56.634295 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:56.634242 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-85d4fbbb6b-7rn5l" podUID="5e97403f-909a-4d0f-bce4-ad9ee3e342bc" containerName="console" containerID="cri-o://fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43" gracePeriod=15 Jul 05 13:20:56.894876 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:56.894825 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:20:57.066653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.066608 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-trusted-ca-bundle\") pod \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " Jul 05 13:20:57.066795 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.066709 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7c65\" (UniqueName: \"kubernetes.io/projected/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-kube-api-access-z7c65\") pod \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " Jul 05 13:20:57.066795 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.066736 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-oauth-config\") pod \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " Jul 05 13:20:57.066795 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.066752 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-service-ca\") pod \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " Jul 05 13:20:57.066795 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.066770 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-serving-cert\") pod \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " Jul 05 13:20:57.066974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.066842 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-oauth-serving-cert\") pod \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " Jul 05 13:20:57.066974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.066878 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-config\") pod \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\" (UID: \"5e97403f-909a-4d0f-bce4-ad9ee3e342bc\") " Jul 05 13:20:57.067222 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.067174 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5e97403f-909a-4d0f-bce4-ad9ee3e342bc" (UID: "5e97403f-909a-4d0f-bce4-ad9ee3e342bc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:57.067222 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.067197 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-service-ca" (OuterVolumeSpecName: "service-ca") pod "5e97403f-909a-4d0f-bce4-ad9ee3e342bc" (UID: "5e97403f-909a-4d0f-bce4-ad9ee3e342bc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:57.067506 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.067280 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5e97403f-909a-4d0f-bce4-ad9ee3e342bc" (UID: "5e97403f-909a-4d0f-bce4-ad9ee3e342bc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:57.067506 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.067417 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-config" (OuterVolumeSpecName: "console-config") pod "5e97403f-909a-4d0f-bce4-ad9ee3e342bc" (UID: "5e97403f-909a-4d0f-bce4-ad9ee3e342bc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:20:57.069099 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.069070 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5e97403f-909a-4d0f-bce4-ad9ee3e342bc" (UID: "5e97403f-909a-4d0f-bce4-ad9ee3e342bc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:57.069187 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.069101 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5e97403f-909a-4d0f-bce4-ad9ee3e342bc" (UID: "5e97403f-909a-4d0f-bce4-ad9ee3e342bc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:20:57.069187 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.069160 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-kube-api-access-z7c65" (OuterVolumeSpecName: "kube-api-access-z7c65") pod "5e97403f-909a-4d0f-bce4-ad9ee3e342bc" (UID: "5e97403f-909a-4d0f-bce4-ad9ee3e342bc"). InnerVolumeSpecName "kube-api-access-z7c65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:20:57.168004 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.167950 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:57.168004 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.167970 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-trusted-ca-bundle\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:57.168004 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.167980 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7c65\" (UniqueName: \"kubernetes.io/projected/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-kube-api-access-z7c65\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:57.168004 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.167988 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-oauth-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:57.168004 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.167998 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-service-ca\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:57.168004 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.168006 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-console-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:57.168241 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.168014 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e97403f-909a-4d0f-bce4-ad9ee3e342bc-oauth-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:20:57.654404 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.654370 2579 generic.go:356] "Generic (PLEG): container finished" podID="5e97403f-909a-4d0f-bce4-ad9ee3e342bc" containerID="fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43" exitCode=0 Jul 05 13:20:57.654810 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.654425 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d4fbbb6b-7rn5l" Jul 05 13:20:57.654810 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.654442 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d4fbbb6b-7rn5l" event={"ID":"5e97403f-909a-4d0f-bce4-ad9ee3e342bc","Type":"ContainerDied","Data":"fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43"} Jul 05 13:20:57.654810 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.654501 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d4fbbb6b-7rn5l" event={"ID":"5e97403f-909a-4d0f-bce4-ad9ee3e342bc","Type":"ContainerDied","Data":"d8ba5874015ce8ee869ef1537d451350724b8a93fb56f27d2c6da5f28547c846"} Jul 05 13:20:57.654810 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.654521 2579 scope.go:117] "RemoveContainer" containerID="fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43" Jul 05 13:20:57.663182 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.663165 2579 scope.go:117] "RemoveContainer" containerID="fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43" Jul 05 13:20:57.663429 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:20:57.663408 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43\": container with ID starting with fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43 not found: ID does not exist" containerID="fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43" Jul 05 13:20:57.663526 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.663435 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43"} err="failed to get container status \"fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43\": rpc error: code = NotFound desc = could not find container \"fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43\": container with ID starting with fe5ce8ea23a351e33a0fe8618fdeea0b2bb6bc66ed2a5e719d4d82479bd97e43 not found: ID does not exist" Jul 05 13:20:57.675461 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.675439 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d4fbbb6b-7rn5l"] Jul 05 13:20:57.683507 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.683487 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85d4fbbb6b-7rn5l"] Jul 05 13:20:57.892847 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:20:57.892817 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e97403f-909a-4d0f-bce4-ad9ee3e342bc" path="/var/lib/kubelet/pods/5e97403f-909a-4d0f-bce4-ad9ee3e342bc/volumes" Jul 05 13:21:22.650481 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:22.650449 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:22.670498 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:22.670473 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:22.763639 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:22.763616 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:40.712146 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:40.712114 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:40.712580 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:40.712378 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="prometheus" containerID="cri-o://352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" gracePeriod=600 Jul 05 13:21:40.712580 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:40.712396 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy" containerID="cri-o://905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" gracePeriod=600 Jul 05 13:21:40.712580 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:40.712425 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="thanos-sidecar" containerID="cri-o://0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" gracePeriod=600 Jul 05 13:21:40.712580 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:40.712472 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" gracePeriod=600 Jul 05 13:21:40.712580 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:40.712484 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy-web" containerID="cri-o://42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" gracePeriod=600 Jul 05 13:21:40.712580 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:40.712427 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="config-reloader" containerID="cri-o://c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" gracePeriod=600 Jul 05 13:21:40.953459 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:40.953434 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:41.097897 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.097859 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-grpc-tls\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.097897 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.097906 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-trusted-ca-bundle\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.097926 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-2\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.097941 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-tls\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098065 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-kube-rbac-proxy\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098110 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098139 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-0\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098167 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzk4z\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-kube-api-access-jzk4z\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098206 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098258 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-tls-assets\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098286 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-serving-certs-ca-bundle\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098287 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:41.098429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098320 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-metrics-client-ca\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098358 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-config\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098381 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-db\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098407 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-1\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098448 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-kubelet-serving-ca-bundle\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098473 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-config-out\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098509 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-metrics-client-certs\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098539 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-web-config\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.098929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098564 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-thanos-prometheus-http-client-file\") pod \"2afb2873-412d-4248-8502-bd783a3fc381\" (UID: \"2afb2873-412d-4248-8502-bd783a3fc381\") " Jul 05 13:21:41.099189 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.098899 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.099789 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.099472 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:41.100259 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.100065 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:41.100599 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.100349 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:41.100939 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.100917 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:21:41.101034 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.100944 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:41.101034 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.100988 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:41.101034 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.101001 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:41.101367 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.101339 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:41.101465 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.101395 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:41.101554 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.101471 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:41.101776 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.101724 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:41.102064 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.101995 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:21:41.102252 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.102227 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:41.102933 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.102910 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:21:41.103280 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.103250 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:41.103371 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.103356 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-kube-api-access-jzk4z" (OuterVolumeSpecName: "kube-api-access-jzk4z") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "kube-api-access-jzk4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:21:41.103496 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.103482 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-config" (OuterVolumeSpecName: "config") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:41.103596 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.103584 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-config-out" (OuterVolumeSpecName: "config-out") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:21:41.113996 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.113975 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-web-config" (OuterVolumeSpecName: "web-config") pod "2afb2873-412d-4248-8502-bd783a3fc381" (UID: "2afb2873-412d-4248-8502-bd783a3fc381"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:21:41.199521 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199478 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199521 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199515 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199521 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199526 2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-kube-rbac-proxy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199521 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199537 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199548 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199558 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzk4z\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-kube-api-access-jzk4z\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199567 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199578 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2afb2873-412d-4248-8502-bd783a3fc381-tls-assets\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199586 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199595 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-metrics-client-ca\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199604 2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199612 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-db\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199621 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199628 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2afb2873-412d-4248-8502-bd783a3fc381-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199639 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2afb2873-412d-4248-8502-bd783a3fc381-config-out\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199648 2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-metrics-client-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199656 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-web-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199664 2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.199813 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.199672 2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2afb2873-412d-4248-8502-bd783a3fc381-secret-grpc-tls\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:21:41.822961 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.822931 2579 generic.go:356] "Generic (PLEG): container finished" podID="2afb2873-412d-4248-8502-bd783a3fc381" containerID="e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" exitCode=0 Jul 05 13:21:41.822961 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.822954 2579 generic.go:356] "Generic (PLEG): container finished" podID="2afb2873-412d-4248-8502-bd783a3fc381" containerID="905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" exitCode=0 Jul 05 13:21:41.822961 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.822960 2579 generic.go:356] "Generic (PLEG): container finished" podID="2afb2873-412d-4248-8502-bd783a3fc381" containerID="42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" exitCode=0 Jul 05 13:21:41.822961 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.822965 2579 generic.go:356] "Generic (PLEG): container finished" podID="2afb2873-412d-4248-8502-bd783a3fc381" containerID="0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" exitCode=0 Jul 05 13:21:41.822961 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.822970 2579 generic.go:356] "Generic (PLEG): container finished" podID="2afb2873-412d-4248-8502-bd783a3fc381" containerID="c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" exitCode=0 Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.822975 2579 generic.go:356] "Generic (PLEG): container finished" podID="2afb2873-412d-4248-8502-bd783a3fc381" containerID="352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" exitCode=0 Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.823035 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerDied","Data":"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d"} Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.823062 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.823125 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerDied","Data":"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940"} Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.823153 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerDied","Data":"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48"} Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.823182 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerDied","Data":"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5"} Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.823194 2579 scope.go:117] "RemoveContainer" containerID="e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.823222 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerDied","Data":"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb"} Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.823260 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerDied","Data":"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc"} Jul 05 13:21:41.823449 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.823304 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2afb2873-412d-4248-8502-bd783a3fc381","Type":"ContainerDied","Data":"c7d24b9b9d24bfd5669ff04def0eaf5cffa480ac18e877809b1a8ffc578be41b"} Jul 05 13:21:41.831076 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.830946 2579 scope.go:117] "RemoveContainer" containerID="905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" Jul 05 13:21:41.838181 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.838165 2579 scope.go:117] "RemoveContainer" containerID="42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" Jul 05 13:21:41.845012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.844994 2579 scope.go:117] "RemoveContainer" containerID="0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" Jul 05 13:21:41.846765 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.846745 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:41.852574 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.852557 2579 scope.go:117] "RemoveContainer" containerID="c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" Jul 05 13:21:41.858258 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.858238 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:41.860828 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.860809 2579 scope.go:117] "RemoveContainer" containerID="352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" Jul 05 13:21:41.867698 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.867659 2579 scope.go:117] "RemoveContainer" containerID="4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075" Jul 05 13:21:41.873885 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.873870 2579 scope.go:117] "RemoveContainer" containerID="e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" Jul 05 13:21:41.874123 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:21:41.874105 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": container with ID starting with e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d not found: ID does not exist" containerID="e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" Jul 05 13:21:41.874187 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.874130 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d"} err="failed to get container status \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": rpc error: code = NotFound desc = could not find container \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": container with ID starting with e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d not found: ID does not exist" Jul 05 13:21:41.874187 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.874147 2579 scope.go:117] "RemoveContainer" containerID="905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" Jul 05 13:21:41.874375 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:21:41.874358 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": container with ID starting with 905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940 not found: ID does not exist" containerID="905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" Jul 05 13:21:41.874417 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.874381 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940"} err="failed to get container status \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": rpc error: code = NotFound desc = could not find container \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": container with ID starting with 905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940 not found: ID does not exist" Jul 05 13:21:41.874417 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.874396 2579 scope.go:117] "RemoveContainer" containerID="42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" Jul 05 13:21:41.874579 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:21:41.874565 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": container with ID starting with 42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48 not found: ID does not exist" containerID="42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" Jul 05 13:21:41.874616 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.874587 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48"} err="failed to get container status \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": rpc error: code = NotFound desc = could not find container \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": container with ID starting with 42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48 not found: ID does not exist" Jul 05 13:21:41.874616 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.874599 2579 scope.go:117] "RemoveContainer" containerID="0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" Jul 05 13:21:41.874792 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:21:41.874777 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": container with ID starting with 0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5 not found: ID does not exist" containerID="0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" Jul 05 13:21:41.874842 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.874794 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5"} err="failed to get container status \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": rpc error: code = NotFound desc = could not find container \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": container with ID starting with 0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5 not found: ID does not exist" Jul 05 13:21:41.874842 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.874806 2579 scope.go:117] "RemoveContainer" containerID="c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" Jul 05 13:21:41.875016 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:21:41.874998 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": container with ID starting with c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb not found: ID does not exist" containerID="c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" Jul 05 13:21:41.875074 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.875024 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb"} err="failed to get container status \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": rpc error: code = NotFound desc = could not find container \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": container with ID starting with c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb not found: ID does not exist" Jul 05 13:21:41.875074 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.875049 2579 scope.go:117] "RemoveContainer" containerID="352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" Jul 05 13:21:41.875262 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:21:41.875240 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": container with ID starting with 352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc not found: ID does not exist" containerID="352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" Jul 05 13:21:41.875326 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.875268 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc"} err="failed to get container status \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": rpc error: code = NotFound desc = could not find container \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": container with ID starting with 352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc not found: ID does not exist" Jul 05 13:21:41.875326 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.875286 2579 scope.go:117] "RemoveContainer" containerID="4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075" Jul 05 13:21:41.875575 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:21:41.875543 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": container with ID starting with 4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075 not found: ID does not exist" containerID="4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075" Jul 05 13:21:41.875575 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.875566 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075"} err="failed to get container status \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": rpc error: code = NotFound desc = could not find container \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": container with ID starting with 4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075 not found: ID does not exist" Jul 05 13:21:41.875770 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.875584 2579 scope.go:117] "RemoveContainer" containerID="e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" Jul 05 13:21:41.875888 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.875867 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d"} err="failed to get container status \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": rpc error: code = NotFound desc = could not find container \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": container with ID starting with e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d not found: ID does not exist" Jul 05 13:21:41.875985 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.875910 2579 scope.go:117] "RemoveContainer" containerID="905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" Jul 05 13:21:41.876171 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.876154 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940"} err="failed to get container status \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": rpc error: code = NotFound desc = could not find container \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": container with ID starting with 905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940 not found: ID does not exist" Jul 05 13:21:41.876220 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.876172 2579 scope.go:117] "RemoveContainer" containerID="42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" Jul 05 13:21:41.876409 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.876388 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48"} err="failed to get container status \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": rpc error: code = NotFound desc = could not find container \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": container with ID starting with 42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48 not found: ID does not exist" Jul 05 13:21:41.876453 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.876410 2579 scope.go:117] "RemoveContainer" containerID="0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" Jul 05 13:21:41.876661 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.876616 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5"} err="failed to get container status \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": rpc error: code = NotFound desc = could not find container \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": container with ID starting with 0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5 not found: ID does not exist" Jul 05 13:21:41.876661 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.876661 2579 scope.go:117] "RemoveContainer" containerID="c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" Jul 05 13:21:41.876917 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.876897 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb"} err="failed to get container status \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": rpc error: code = NotFound desc = could not find container \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": container with ID starting with c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb not found: ID does not exist" Jul 05 13:21:41.876998 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.876919 2579 scope.go:117] "RemoveContainer" containerID="352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" Jul 05 13:21:41.877153 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877127 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc"} err="failed to get container status \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": rpc error: code = NotFound desc = could not find container \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": container with ID starting with 352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc not found: ID does not exist" Jul 05 13:21:41.877197 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877154 2579 scope.go:117] "RemoveContainer" containerID="4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075" Jul 05 13:21:41.877359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877340 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075"} err="failed to get container status \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": rpc error: code = NotFound desc = could not find container \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": container with ID starting with 4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075 not found: ID does not exist" Jul 05 13:21:41.877400 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877361 2579 scope.go:117] "RemoveContainer" containerID="e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" Jul 05 13:21:41.877562 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877545 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d"} err="failed to get container status \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": rpc error: code = NotFound desc = could not find container \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": container with ID starting with e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d not found: ID does not exist" Jul 05 13:21:41.877607 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877562 2579 scope.go:117] "RemoveContainer" containerID="905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" Jul 05 13:21:41.877752 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877737 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940"} err="failed to get container status \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": rpc error: code = NotFound desc = could not find container \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": container with ID starting with 905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940 not found: ID does not exist" Jul 05 13:21:41.877804 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877752 2579 scope.go:117] "RemoveContainer" containerID="42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" Jul 05 13:21:41.877957 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877933 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48"} err="failed to get container status \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": rpc error: code = NotFound desc = could not find container \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": container with ID starting with 42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48 not found: ID does not exist" Jul 05 13:21:41.877957 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.877956 2579 scope.go:117] "RemoveContainer" containerID="0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" Jul 05 13:21:41.878132 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878117 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5"} err="failed to get container status \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": rpc error: code = NotFound desc = could not find container \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": container with ID starting with 0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5 not found: ID does not exist" Jul 05 13:21:41.878177 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878132 2579 scope.go:117] "RemoveContainer" containerID="c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" Jul 05 13:21:41.878310 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878295 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb"} err="failed to get container status \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": rpc error: code = NotFound desc = could not find container \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": container with ID starting with c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb not found: ID does not exist" Jul 05 13:21:41.878350 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878311 2579 scope.go:117] "RemoveContainer" containerID="352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" Jul 05 13:21:41.878485 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878466 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc"} err="failed to get container status \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": rpc error: code = NotFound desc = could not find container \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": container with ID starting with 352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc not found: ID does not exist" Jul 05 13:21:41.878527 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878485 2579 scope.go:117] "RemoveContainer" containerID="4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075" Jul 05 13:21:41.878670 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878652 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075"} err="failed to get container status \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": rpc error: code = NotFound desc = could not find container \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": container with ID starting with 4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075 not found: ID does not exist" Jul 05 13:21:41.878770 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878672 2579 scope.go:117] "RemoveContainer" containerID="e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" Jul 05 13:21:41.878926 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878910 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d"} err="failed to get container status \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": rpc error: code = NotFound desc = could not find container \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": container with ID starting with e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d not found: ID does not exist" Jul 05 13:21:41.878987 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.878926 2579 scope.go:117] "RemoveContainer" containerID="905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" Jul 05 13:21:41.879088 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879073 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940"} err="failed to get container status \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": rpc error: code = NotFound desc = could not find container \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": container with ID starting with 905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940 not found: ID does not exist" Jul 05 13:21:41.879140 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879088 2579 scope.go:117] "RemoveContainer" containerID="42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" Jul 05 13:21:41.879247 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879229 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48"} err="failed to get container status \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": rpc error: code = NotFound desc = could not find container \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": container with ID starting with 42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48 not found: ID does not exist" Jul 05 13:21:41.879316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879249 2579 scope.go:117] "RemoveContainer" containerID="0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" Jul 05 13:21:41.879416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879401 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5"} err="failed to get container status \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": rpc error: code = NotFound desc = could not find container \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": container with ID starting with 0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5 not found: ID does not exist" Jul 05 13:21:41.879466 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879417 2579 scope.go:117] "RemoveContainer" containerID="c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" Jul 05 13:21:41.879586 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879571 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb"} err="failed to get container status \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": rpc error: code = NotFound desc = could not find container \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": container with ID starting with c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb not found: ID does not exist" Jul 05 13:21:41.879640 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879587 2579 scope.go:117] "RemoveContainer" containerID="352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" Jul 05 13:21:41.879763 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879749 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc"} err="failed to get container status \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": rpc error: code = NotFound desc = could not find container \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": container with ID starting with 352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc not found: ID does not exist" Jul 05 13:21:41.879815 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879763 2579 scope.go:117] "RemoveContainer" containerID="4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075" Jul 05 13:21:41.879929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879916 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075"} err="failed to get container status \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": rpc error: code = NotFound desc = could not find container \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": container with ID starting with 4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075 not found: ID does not exist" Jul 05 13:21:41.879974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.879929 2579 scope.go:117] "RemoveContainer" containerID="e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" Jul 05 13:21:41.880128 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880110 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d"} err="failed to get container status \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": rpc error: code = NotFound desc = could not find container \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": container with ID starting with e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d not found: ID does not exist" Jul 05 13:21:41.880173 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880131 2579 scope.go:117] "RemoveContainer" containerID="905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" Jul 05 13:21:41.880319 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880304 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940"} err="failed to get container status \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": rpc error: code = NotFound desc = could not find container \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": container with ID starting with 905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940 not found: ID does not exist" Jul 05 13:21:41.880361 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880318 2579 scope.go:117] "RemoveContainer" containerID="42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" Jul 05 13:21:41.880489 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880465 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48"} err="failed to get container status \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": rpc error: code = NotFound desc = could not find container \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": container with ID starting with 42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48 not found: ID does not exist" Jul 05 13:21:41.880534 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880489 2579 scope.go:117] "RemoveContainer" containerID="0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" Jul 05 13:21:41.880646 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880628 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5"} err="failed to get container status \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": rpc error: code = NotFound desc = could not find container \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": container with ID starting with 0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5 not found: ID does not exist" Jul 05 13:21:41.880739 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880648 2579 scope.go:117] "RemoveContainer" containerID="c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" Jul 05 13:21:41.880887 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880873 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb"} err="failed to get container status \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": rpc error: code = NotFound desc = could not find container \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": container with ID starting with c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb not found: ID does not exist" Jul 05 13:21:41.880944 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.880888 2579 scope.go:117] "RemoveContainer" containerID="352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" Jul 05 13:21:41.881050 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881034 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc"} err="failed to get container status \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": rpc error: code = NotFound desc = could not find container \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": container with ID starting with 352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc not found: ID does not exist" Jul 05 13:21:41.881094 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881051 2579 scope.go:117] "RemoveContainer" containerID="4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075" Jul 05 13:21:41.881266 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881250 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075"} err="failed to get container status \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": rpc error: code = NotFound desc = could not find container \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": container with ID starting with 4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075 not found: ID does not exist" Jul 05 13:21:41.881316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881268 2579 scope.go:117] "RemoveContainer" containerID="e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d" Jul 05 13:21:41.881428 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881413 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d"} err="failed to get container status \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": rpc error: code = NotFound desc = could not find container \"e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d\": container with ID starting with e7df4e26d21960f43c73d7845527afdd3c6b652ec2d08df9ad4b56702356375d not found: ID does not exist" Jul 05 13:21:41.881485 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881431 2579 scope.go:117] "RemoveContainer" containerID="905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940" Jul 05 13:21:41.881634 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881616 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940"} err="failed to get container status \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": rpc error: code = NotFound desc = could not find container \"905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940\": container with ID starting with 905944d29b7ba031070b39028b7a68308388d30fce2c52485e8a0e2bc7a59940 not found: ID does not exist" Jul 05 13:21:41.881736 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881635 2579 scope.go:117] "RemoveContainer" containerID="42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48" Jul 05 13:21:41.881854 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881837 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48"} err="failed to get container status \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": rpc error: code = NotFound desc = could not find container \"42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48\": container with ID starting with 42099aae3655e0494275756e3d96ca92367405c778ece423e8baed310eda3a48 not found: ID does not exist" Jul 05 13:21:41.881900 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.881856 2579 scope.go:117] "RemoveContainer" containerID="0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5" Jul 05 13:21:41.882055 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.882041 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5"} err="failed to get container status \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": rpc error: code = NotFound desc = could not find container \"0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5\": container with ID starting with 0f91adb2e0f3c405fdaaaf2352f1dc39e847dffbc2e375ec09a74b670f4b9ec5 not found: ID does not exist" Jul 05 13:21:41.882055 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.882055 2579 scope.go:117] "RemoveContainer" containerID="c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb" Jul 05 13:21:41.882248 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.882232 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb"} err="failed to get container status \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": rpc error: code = NotFound desc = could not find container \"c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb\": container with ID starting with c7f361b3176ee3f1ae350d0ffd344d99e287a9eac7f475f301059bfc1e698bbb not found: ID does not exist" Jul 05 13:21:41.882292 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.882248 2579 scope.go:117] "RemoveContainer" containerID="352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc" Jul 05 13:21:41.882439 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.882422 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc"} err="failed to get container status \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": rpc error: code = NotFound desc = could not find container \"352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc\": container with ID starting with 352e93f0cfad87e0fe563f859bfe5cd88cc19db8983e7e6e9b559cd9d5ed5dfc not found: ID does not exist" Jul 05 13:21:41.882482 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.882439 2579 scope.go:117] "RemoveContainer" containerID="4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075" Jul 05 13:21:41.882606 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.882591 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075"} err="failed to get container status \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": rpc error: code = NotFound desc = could not find container \"4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075\": container with ID starting with 4f7491af845bb6060be74edcdcb8063165b9e6b93a47989a14177cfa838fc075 not found: ID does not exist" Jul 05 13:21:41.885110 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885089 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:41.885359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885347 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="prometheus" Jul 05 13:21:41.885401 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885361 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="prometheus" Jul 05 13:21:41.885401 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885371 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6c7f18b-e361-4fa5-949e-60f69bcbb880" containerName="registry" Jul 05 13:21:41.885401 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885376 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6c7f18b-e361-4fa5-949e-60f69bcbb880" containerName="registry" Jul 05 13:21:41.885401 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885384 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy" Jul 05 13:21:41.885401 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885390 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy" Jul 05 13:21:41.885401 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885396 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e97403f-909a-4d0f-bce4-ad9ee3e342bc" containerName="console" Jul 05 13:21:41.885401 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885401 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e97403f-909a-4d0f-bce4-ad9ee3e342bc" containerName="console" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885408 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="init-config-reloader" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885415 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="init-config-reloader" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885425 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41cad192-9110-4a08-b9e2-a629c982f5ff" containerName="console" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885430 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41cad192-9110-4a08-b9e2-a629c982f5ff" containerName="console" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885438 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="thanos-sidecar" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885443 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="thanos-sidecar" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885449 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="config-reloader" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885454 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="config-reloader" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885459 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy-thanos" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885463 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy-thanos" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885471 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy-web" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885476 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy-web" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885515 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41cad192-9110-4a08-b9e2-a629c982f5ff" containerName="console" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885522 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e97403f-909a-4d0f-bce4-ad9ee3e342bc" containerName="console" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885529 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885534 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6c7f18b-e361-4fa5-949e-60f69bcbb880" containerName="registry" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885539 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="config-reloader" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885544 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="thanos-sidecar" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885550 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="prometheus" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885555 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy-web" Jul 05 13:21:41.885628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.885560 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2afb2873-412d-4248-8502-bd783a3fc381" containerName="kube-rbac-proxy-thanos" Jul 05 13:21:41.890803 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.890784 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:41.893258 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.893240 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 05 13:21:41.893355 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.893325 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 05 13:21:41.893355 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.893243 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 05 13:21:41.893471 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.893405 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 05 13:21:41.893528 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.893484 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 05 13:21:41.893528 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.893504 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 05 13:21:41.893720 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.893668 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 05 13:21:41.893803 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.893735 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 05 13:21:41.893865 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.893846 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-t7jfj\"" Jul 05 13:21:41.894239 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.894223 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 05 13:21:41.894608 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.894581 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 05 13:21:41.894719 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.894610 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 05 13:21:41.894784 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.894737 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 05 13:21:41.894841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.894784 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-br2ukfr548p9j\"" Jul 05 13:21:41.894841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.894801 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 05 13:21:41.896989 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.896971 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 05 13:21:41.897207 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.897187 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2afb2873-412d-4248-8502-bd783a3fc381" path="/var/lib/kubelet/pods/2afb2873-412d-4248-8502-bd783a3fc381/volumes" Jul 05 13:21:41.899359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.899232 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 05 13:21:41.904021 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:41.904003 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:42.004929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.004866 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.004929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.004896 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.004929 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.004927 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.004945 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.004961 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr2b4\" (UniqueName: \"kubernetes.io/projected/0e74bc18-36b5-476f-b2f7-411b94eae2f6-kube-api-access-pr2b4\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005045 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005063 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005082 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005398 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005137 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-config\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005398 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005398 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005199 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e74bc18-36b5-476f-b2f7-411b94eae2f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005398 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005257 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005398 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005398 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005339 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005398 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005364 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005592 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005507 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e74bc18-36b5-476f-b2f7-411b94eae2f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005592 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005529 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005592 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005555 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.005592 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.005569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106539 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106515 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106544 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106563 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106579 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106598 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e74bc18-36b5-476f-b2f7-411b94eae2f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106614 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106632 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106708 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106762 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106785 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106806 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106827 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr2b4\" (UniqueName: \"kubernetes.io/projected/0e74bc18-36b5-476f-b2f7-411b94eae2f6-kube-api-access-pr2b4\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.106914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106910 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.107545 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106939 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-config\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.107545 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106961 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.107545 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.106990 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e74bc18-36b5-476f-b2f7-411b94eae2f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.110348 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.109634 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.110348 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.109642 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0e74bc18-36b5-476f-b2f7-411b94eae2f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.110348 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.109746 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0e74bc18-36b5-476f-b2f7-411b94eae2f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.110348 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.109840 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.110348 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.109999 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.111335 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.110547 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.111335 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.110589 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.111335 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.110809 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.111335 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.110857 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.111335 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.111233 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.111592 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.111422 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.112151 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.112127 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.112231 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.112181 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e74bc18-36b5-476f-b2f7-411b94eae2f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.112653 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.112627 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.112749 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.112657 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.112749 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.112636 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.112822 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.112755 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.113101 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.113079 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.113397 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.113379 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0e74bc18-36b5-476f-b2f7-411b94eae2f6-config\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.117509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.117490 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr2b4\" (UniqueName: \"kubernetes.io/projected/0e74bc18-36b5-476f-b2f7-411b94eae2f6-kube-api-access-pr2b4\") pod \"prometheus-k8s-0\" (UID: \"0e74bc18-36b5-476f-b2f7-411b94eae2f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.201629 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.201599 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:42.343122 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.343098 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 13:21:42.345551 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:21:42.345523 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e74bc18_36b5_476f_b2f7_411b94eae2f6.slice/crio-894f62323992f3e003029544189c6538ad66ef8bc3be26289bdf6482c6828173 WatchSource:0}: Error finding container 894f62323992f3e003029544189c6538ad66ef8bc3be26289bdf6482c6828173: Status 404 returned error can't find the container with id 894f62323992f3e003029544189c6538ad66ef8bc3be26289bdf6482c6828173 Jul 05 13:21:42.832213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.832174 2579 generic.go:356] "Generic (PLEG): container finished" podID="0e74bc18-36b5-476f-b2f7-411b94eae2f6" containerID="406f7fd47c298443144866dd00c86a3e90fb80f2d08457b99ae1b00257c6c872" exitCode=0 Jul 05 13:21:42.832634 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.832259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e74bc18-36b5-476f-b2f7-411b94eae2f6","Type":"ContainerDied","Data":"406f7fd47c298443144866dd00c86a3e90fb80f2d08457b99ae1b00257c6c872"} Jul 05 13:21:42.832634 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:42.832375 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e74bc18-36b5-476f-b2f7-411b94eae2f6","Type":"ContainerStarted","Data":"894f62323992f3e003029544189c6538ad66ef8bc3be26289bdf6482c6828173"} Jul 05 13:21:43.839697 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:43.839647 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e74bc18-36b5-476f-b2f7-411b94eae2f6","Type":"ContainerStarted","Data":"eeeb704745a4efb6cec72fd88cf692487591ab07a1c22f7df374ee3b9e46d465"} Jul 05 13:21:43.840119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:43.839721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e74bc18-36b5-476f-b2f7-411b94eae2f6","Type":"ContainerStarted","Data":"6e4990baa3216bd04d5ac3db658c37d2353d814e6072d8c1564866076aefbd34"} Jul 05 13:21:43.840119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:43.839747 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e74bc18-36b5-476f-b2f7-411b94eae2f6","Type":"ContainerStarted","Data":"f641c098b4a4b6820d7923014a0734ba295b51a87d44138cb1bf82142ca76461"} Jul 05 13:21:43.840119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:43.839771 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e74bc18-36b5-476f-b2f7-411b94eae2f6","Type":"ContainerStarted","Data":"90a44f1b61c41138544379e3334f2a599277029ff8d0c11df72a0b2ca3b210be"} Jul 05 13:21:43.840119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:43.839794 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e74bc18-36b5-476f-b2f7-411b94eae2f6","Type":"ContainerStarted","Data":"83cf452a16a94a273a2462122d0d2c0d339e1c2955952831fa7e12d78aaba05d"} Jul 05 13:21:43.840119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:43.839816 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0e74bc18-36b5-476f-b2f7-411b94eae2f6","Type":"ContainerStarted","Data":"4420f5a2c1980612f868040e5b82259c5dc4a7391c47cf4f18682ded8295245d"} Jul 05 13:21:43.867781 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:43.867737 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.849116046 podStartE2EDuration="2.867720505s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.604459ms" isStatefulPod=false podCreationTimestamp="2026-07-05 13:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:21:43.866617333 +0000 UTC m=+208.494983088" watchObservedRunningTime="2026-07-05 13:21:43.867720505 +0000 UTC m=+208.496086261" Jul 05 13:21:43.997054 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:43.997026 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68b5f4b475-phkcd"] Jul 05 13:21:44.001379 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.001361 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.015958 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.015936 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68b5f4b475-phkcd"] Jul 05 13:21:44.125760 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.125674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-service-ca\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.125760 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.125731 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-console-config\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.125760 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.125751 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-trusted-ca-bundle\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.125980 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.125769 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-oauth-serving-cert\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.125980 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.125790 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz4g9\" (UniqueName: \"kubernetes.io/projected/27b00532-bcb7-4121-a541-bd04837f2336-kube-api-access-xz4g9\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.125980 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.125813 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-serving-cert\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.125980 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.125832 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-oauth-config\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227009 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.226980 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-service-ca\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227023 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-console-config\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227050 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-trusted-ca-bundle\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-oauth-serving-cert\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227124 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz4g9\" (UniqueName: \"kubernetes.io/projected/27b00532-bcb7-4121-a541-bd04837f2336-kube-api-access-xz4g9\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227372 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-serving-cert\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227372 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227176 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-oauth-config\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227809 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227784 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-console-config\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227911 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227803 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-service-ca\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227960 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227905 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-oauth-serving-cert\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.227999 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.227979 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-trusted-ca-bundle\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.229593 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.229574 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-serving-cert\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.229652 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.229634 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-oauth-config\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.235487 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.235457 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz4g9\" (UniqueName: \"kubernetes.io/projected/27b00532-bcb7-4121-a541-bd04837f2336-kube-api-access-xz4g9\") pod \"console-68b5f4b475-phkcd\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.310287 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.310262 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:44.439398 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.439308 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68b5f4b475-phkcd"] Jul 05 13:21:44.441411 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:21:44.441388 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b00532_bcb7_4121_a541_bd04837f2336.slice/crio-9a05c8f461f9589a7d6f7ca8368bcd2dba1f2fc5a5663db5d9d6ae88a0b0d6f0 WatchSource:0}: Error finding container 9a05c8f461f9589a7d6f7ca8368bcd2dba1f2fc5a5663db5d9d6ae88a0b0d6f0: Status 404 returned error can't find the container with id 9a05c8f461f9589a7d6f7ca8368bcd2dba1f2fc5a5663db5d9d6ae88a0b0d6f0 Jul 05 13:21:44.845017 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.844982 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68b5f4b475-phkcd" event={"ID":"27b00532-bcb7-4121-a541-bd04837f2336","Type":"ContainerStarted","Data":"d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22"} Jul 05 13:21:44.845363 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.845037 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68b5f4b475-phkcd" event={"ID":"27b00532-bcb7-4121-a541-bd04837f2336","Type":"ContainerStarted","Data":"9a05c8f461f9589a7d6f7ca8368bcd2dba1f2fc5a5663db5d9d6ae88a0b0d6f0"} Jul 05 13:21:44.864046 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:44.863990 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68b5f4b475-phkcd" podStartSLOduration=1.863969683 podStartE2EDuration="1.863969683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:21:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:21:44.863332636 +0000 UTC m=+209.491698382" watchObservedRunningTime="2026-07-05 13:21:44.863969683 +0000 UTC m=+209.492335439" Jul 05 13:21:47.201967 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:47.201930 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:21:54.310826 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:54.310777 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:54.311294 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:54.310847 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:54.315566 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:54.315539 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:54.890123 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:54.890096 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:21:54.937059 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:21:54.937025 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65fcdddf4b-x65dd"] Jul 05 13:22:19.959100 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:19.959037 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-65fcdddf4b-x65dd" podUID="2c053694-e213-4fd7-b60e-e9170ee60a5a" containerName="console" containerID="cri-o://c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c" gracePeriod=15 Jul 05 13:22:20.204395 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.204369 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:22:20.310158 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310126 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-serving-cert\") pod \"2c053694-e213-4fd7-b60e-e9170ee60a5a\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " Jul 05 13:22:20.310158 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310162 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-oauth-config\") pod \"2c053694-e213-4fd7-b60e-e9170ee60a5a\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " Jul 05 13:22:20.310459 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310186 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-trusted-ca-bundle\") pod \"2c053694-e213-4fd7-b60e-e9170ee60a5a\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " Jul 05 13:22:20.310459 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310223 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-service-ca\") pod \"2c053694-e213-4fd7-b60e-e9170ee60a5a\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " Jul 05 13:22:20.310459 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310269 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4rht\" (UniqueName: \"kubernetes.io/projected/2c053694-e213-4fd7-b60e-e9170ee60a5a-kube-api-access-l4rht\") pod \"2c053694-e213-4fd7-b60e-e9170ee60a5a\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " Jul 05 13:22:20.310459 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310299 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-oauth-serving-cert\") pod \"2c053694-e213-4fd7-b60e-e9170ee60a5a\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " Jul 05 13:22:20.310459 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310342 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-config\") pod \"2c053694-e213-4fd7-b60e-e9170ee60a5a\" (UID: \"2c053694-e213-4fd7-b60e-e9170ee60a5a\") " Jul 05 13:22:20.310741 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310653 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2c053694-e213-4fd7-b60e-e9170ee60a5a" (UID: "2c053694-e213-4fd7-b60e-e9170ee60a5a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:20.310826 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310716 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-service-ca" (OuterVolumeSpecName: "service-ca") pod "2c053694-e213-4fd7-b60e-e9170ee60a5a" (UID: "2c053694-e213-4fd7-b60e-e9170ee60a5a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:20.310886 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310863 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-config" (OuterVolumeSpecName: "console-config") pod "2c053694-e213-4fd7-b60e-e9170ee60a5a" (UID: "2c053694-e213-4fd7-b60e-e9170ee60a5a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:20.311107 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.310892 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2c053694-e213-4fd7-b60e-e9170ee60a5a" (UID: "2c053694-e213-4fd7-b60e-e9170ee60a5a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:22:20.312520 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.312489 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2c053694-e213-4fd7-b60e-e9170ee60a5a" (UID: "2c053694-e213-4fd7-b60e-e9170ee60a5a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:22:20.313106 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.313078 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c053694-e213-4fd7-b60e-e9170ee60a5a-kube-api-access-l4rht" (OuterVolumeSpecName: "kube-api-access-l4rht") pod "2c053694-e213-4fd7-b60e-e9170ee60a5a" (UID: "2c053694-e213-4fd7-b60e-e9170ee60a5a"). InnerVolumeSpecName "kube-api-access-l4rht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:22:20.313196 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.313126 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2c053694-e213-4fd7-b60e-e9170ee60a5a" (UID: "2c053694-e213-4fd7-b60e-e9170ee60a5a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:22:20.411751 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.411712 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-trusted-ca-bundle\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:22:20.411751 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.411740 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-service-ca\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:22:20.411751 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.411750 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4rht\" (UniqueName: \"kubernetes.io/projected/2c053694-e213-4fd7-b60e-e9170ee60a5a-kube-api-access-l4rht\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:22:20.411751 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.411760 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-oauth-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:22:20.412101 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.411768 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:22:20.412101 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.411776 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:22:20.412101 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.411784 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2c053694-e213-4fd7-b60e-e9170ee60a5a-console-oauth-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:22:20.985408 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.985375 2579 generic.go:356] "Generic (PLEG): container finished" podID="2c053694-e213-4fd7-b60e-e9170ee60a5a" containerID="c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c" exitCode=0 Jul 05 13:22:20.985408 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.985414 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65fcdddf4b-x65dd" event={"ID":"2c053694-e213-4fd7-b60e-e9170ee60a5a","Type":"ContainerDied","Data":"c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c"} Jul 05 13:22:20.985844 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.985433 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65fcdddf4b-x65dd" Jul 05 13:22:20.985844 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.985453 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65fcdddf4b-x65dd" event={"ID":"2c053694-e213-4fd7-b60e-e9170ee60a5a","Type":"ContainerDied","Data":"420a337001024f0add5cbca9691c81bce7b209d4dcca9143fb97d779998f7d5d"} Jul 05 13:22:20.985844 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.985472 2579 scope.go:117] "RemoveContainer" containerID="c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c" Jul 05 13:22:20.994359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.994342 2579 scope.go:117] "RemoveContainer" containerID="c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c" Jul 05 13:22:20.994610 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:22:20.994588 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c\": container with ID starting with c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c not found: ID does not exist" containerID="c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c" Jul 05 13:22:20.994691 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:20.994620 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c"} err="failed to get container status \"c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c\": rpc error: code = NotFound desc = could not find container \"c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c\": container with ID starting with c046dd53cb6e1ec33b6cd7f7958b2496ddb82528e0d9db1353fc34611f156a6c not found: ID does not exist" Jul 05 13:22:21.008791 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:21.008765 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65fcdddf4b-x65dd"] Jul 05 13:22:21.015319 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:21.015299 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65fcdddf4b-x65dd"] Jul 05 13:22:21.892247 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:21.892219 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c053694-e213-4fd7-b60e-e9170ee60a5a" path="/var/lib/kubelet/pods/2c053694-e213-4fd7-b60e-e9170ee60a5a/volumes" Jul 05 13:22:42.202444 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:42.202348 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:22:42.218351 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:42.218314 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:22:43.083218 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:43.083188 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 13:22:54.344112 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.344082 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57fdb5d6bd-njwbs"] Jul 05 13:22:54.344562 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.344363 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c053694-e213-4fd7-b60e-e9170ee60a5a" containerName="console" Jul 05 13:22:54.344562 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.344375 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c053694-e213-4fd7-b60e-e9170ee60a5a" containerName="console" Jul 05 13:22:54.344562 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.344420 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c053694-e213-4fd7-b60e-e9170ee60a5a" containerName="console" Jul 05 13:22:54.347354 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.347337 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.362623 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.362602 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57fdb5d6bd-njwbs"] Jul 05 13:22:54.491469 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.491437 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvrqw\" (UniqueName: \"kubernetes.io/projected/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-kube-api-access-nvrqw\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.491628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.491480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-service-ca\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.491628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.491552 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-config\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.491628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.491570 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-trusted-ca-bundle\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.491761 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.491643 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-serving-cert\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.491761 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.491669 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-oauth-config\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.491761 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.491706 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-oauth-serving-cert\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.592089 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.592053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-serving-cert\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.592089 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.592093 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-oauth-config\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.592279 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.592114 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-oauth-serving-cert\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.592279 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.592157 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvrqw\" (UniqueName: \"kubernetes.io/projected/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-kube-api-access-nvrqw\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.592279 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.592195 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-service-ca\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.592279 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.592260 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-config\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.592469 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.592284 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-trusted-ca-bundle\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.592949 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.592924 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-oauth-serving-cert\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.593056 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.592951 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-service-ca\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.593236 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.593213 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-config\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.593588 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.593567 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-trusted-ca-bundle\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.594734 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.594672 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-oauth-config\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.594938 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.594919 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-serving-cert\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.604217 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.604186 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvrqw\" (UniqueName: \"kubernetes.io/projected/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-kube-api-access-nvrqw\") pod \"console-57fdb5d6bd-njwbs\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.655592 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.655560 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:22:54.819007 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:54.818971 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57fdb5d6bd-njwbs"] Jul 05 13:22:54.823442 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:22:54.823415 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b5273f_ddbc_423c_b5f3_c31b1bc248ae.slice/crio-a1d7556edc18012867baa7f146bb9ffafa7cd41da71aad732c6e87de927ad257 WatchSource:0}: Error finding container a1d7556edc18012867baa7f146bb9ffafa7cd41da71aad732c6e87de927ad257: Status 404 returned error can't find the container with id a1d7556edc18012867baa7f146bb9ffafa7cd41da71aad732c6e87de927ad257 Jul 05 13:22:55.114770 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:55.114667 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fdb5d6bd-njwbs" event={"ID":"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae","Type":"ContainerStarted","Data":"bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861"} Jul 05 13:22:55.114770 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:55.114737 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fdb5d6bd-njwbs" event={"ID":"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae","Type":"ContainerStarted","Data":"a1d7556edc18012867baa7f146bb9ffafa7cd41da71aad732c6e87de927ad257"} Jul 05 13:22:55.136445 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:22:55.136391 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57fdb5d6bd-njwbs" podStartSLOduration=1.1363751 podStartE2EDuration="1.1363751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:22:55.134733959 +0000 UTC m=+279.763099722" watchObservedRunningTime="2026-07-05 13:22:55.1363751 +0000 UTC m=+279.764740855" Jul 05 13:23:04.656696 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:04.656647 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:23:04.657099 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:04.656741 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:23:04.662428 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:04.662405 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:23:05.155580 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:05.155555 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:23:05.205365 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:05.205336 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68b5f4b475-phkcd"] Jul 05 13:23:15.781646 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:15.781623 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 05 13:23:30.227157 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.227108 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-68b5f4b475-phkcd" podUID="27b00532-bcb7-4121-a541-bd04837f2336" containerName="console" containerID="cri-o://d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22" gracePeriod=15 Jul 05 13:23:30.462809 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.462785 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:23:30.570659 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.570629 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-console-config\") pod \"27b00532-bcb7-4121-a541-bd04837f2336\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " Jul 05 13:23:30.570832 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.570698 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-oauth-serving-cert\") pod \"27b00532-bcb7-4121-a541-bd04837f2336\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " Jul 05 13:23:30.570832 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.570729 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-service-ca\") pod \"27b00532-bcb7-4121-a541-bd04837f2336\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " Jul 05 13:23:30.570966 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.570845 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-trusted-ca-bundle\") pod \"27b00532-bcb7-4121-a541-bd04837f2336\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " Jul 05 13:23:30.570966 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.570907 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xz4g9\" (UniqueName: \"kubernetes.io/projected/27b00532-bcb7-4121-a541-bd04837f2336-kube-api-access-xz4g9\") pod \"27b00532-bcb7-4121-a541-bd04837f2336\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " Jul 05 13:23:30.570966 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.570941 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-oauth-config\") pod \"27b00532-bcb7-4121-a541-bd04837f2336\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " Jul 05 13:23:30.571113 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.570974 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-serving-cert\") pod \"27b00532-bcb7-4121-a541-bd04837f2336\" (UID: \"27b00532-bcb7-4121-a541-bd04837f2336\") " Jul 05 13:23:30.571113 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.571004 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-console-config" (OuterVolumeSpecName: "console-config") pod "27b00532-bcb7-4121-a541-bd04837f2336" (UID: "27b00532-bcb7-4121-a541-bd04837f2336"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:23:30.571203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.571149 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "27b00532-bcb7-4121-a541-bd04837f2336" (UID: "27b00532-bcb7-4121-a541-bd04837f2336"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:23:30.571203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.571157 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-service-ca" (OuterVolumeSpecName: "service-ca") pod "27b00532-bcb7-4121-a541-bd04837f2336" (UID: "27b00532-bcb7-4121-a541-bd04837f2336"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:23:30.571286 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.571222 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "27b00532-bcb7-4121-a541-bd04837f2336" (UID: "27b00532-bcb7-4121-a541-bd04837f2336"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:23:30.571286 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.571236 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-console-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:23:30.571286 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.571254 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-oauth-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:23:30.571286 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.571265 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-service-ca\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:23:30.573191 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.573163 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "27b00532-bcb7-4121-a541-bd04837f2336" (UID: "27b00532-bcb7-4121-a541-bd04837f2336"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:23:30.573191 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.573178 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "27b00532-bcb7-4121-a541-bd04837f2336" (UID: "27b00532-bcb7-4121-a541-bd04837f2336"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:23:30.573318 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.573229 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27b00532-bcb7-4121-a541-bd04837f2336-kube-api-access-xz4g9" (OuterVolumeSpecName: "kube-api-access-xz4g9") pod "27b00532-bcb7-4121-a541-bd04837f2336" (UID: "27b00532-bcb7-4121-a541-bd04837f2336"). InnerVolumeSpecName "kube-api-access-xz4g9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:23:30.672328 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.672293 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xz4g9\" (UniqueName: \"kubernetes.io/projected/27b00532-bcb7-4121-a541-bd04837f2336-kube-api-access-xz4g9\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:23:30.672328 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.672321 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-oauth-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:23:30.672328 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.672331 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27b00532-bcb7-4121-a541-bd04837f2336-console-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:23:30.672555 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:30.672341 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27b00532-bcb7-4121-a541-bd04837f2336-trusted-ca-bundle\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:23:31.254713 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.254661 2579 generic.go:356] "Generic (PLEG): container finished" podID="27b00532-bcb7-4121-a541-bd04837f2336" containerID="d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22" exitCode=0 Jul 05 13:23:31.255159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.254731 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68b5f4b475-phkcd" event={"ID":"27b00532-bcb7-4121-a541-bd04837f2336","Type":"ContainerDied","Data":"d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22"} Jul 05 13:23:31.255159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.254744 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68b5f4b475-phkcd" Jul 05 13:23:31.255159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.254773 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68b5f4b475-phkcd" event={"ID":"27b00532-bcb7-4121-a541-bd04837f2336","Type":"ContainerDied","Data":"9a05c8f461f9589a7d6f7ca8368bcd2dba1f2fc5a5663db5d9d6ae88a0b0d6f0"} Jul 05 13:23:31.255159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.254792 2579 scope.go:117] "RemoveContainer" containerID="d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22" Jul 05 13:23:31.263162 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.263146 2579 scope.go:117] "RemoveContainer" containerID="d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22" Jul 05 13:23:31.263410 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:23:31.263391 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22\": container with ID starting with d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22 not found: ID does not exist" containerID="d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22" Jul 05 13:23:31.263470 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.263417 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22"} err="failed to get container status \"d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22\": rpc error: code = NotFound desc = could not find container \"d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22\": container with ID starting with d031f51acaef229177d15f39c46c33d65c06a034c05c75fb5e8a8c47a9e9fd22 not found: ID does not exist" Jul 05 13:23:31.276174 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.276149 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68b5f4b475-phkcd"] Jul 05 13:23:31.284110 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.284088 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68b5f4b475-phkcd"] Jul 05 13:23:31.892313 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:23:31.892281 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27b00532-bcb7-4121-a541-bd04837f2336" path="/var/lib/kubelet/pods/27b00532-bcb7-4121-a541-bd04837f2336/volumes" Jul 05 13:24:36.996613 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:36.996579 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2rvsr"] Jul 05 13:24:36.997081 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:36.996879 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27b00532-bcb7-4121-a541-bd04837f2336" containerName="console" Jul 05 13:24:36.997081 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:36.996891 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="27b00532-bcb7-4121-a541-bd04837f2336" containerName="console" Jul 05 13:24:36.997081 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:36.996949 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="27b00532-bcb7-4121-a541-bd04837f2336" containerName="console" Jul 05 13:24:36.999747 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:36.999731 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:24:37.006321 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.006301 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 05 13:24:37.007371 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.007341 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-9jwhp\"" Jul 05 13:24:37.007473 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.007411 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 05 13:24:37.055475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.055451 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36dea70-20ce-4185-ac0d-72469bed6792-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2rvsr\" (UID: \"f36dea70-20ce-4185-ac0d-72469bed6792\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:24:37.055577 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.055517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2bnn\" (UniqueName: \"kubernetes.io/projected/f36dea70-20ce-4185-ac0d-72469bed6792-kube-api-access-z2bnn\") pod \"cert-manager-webhook-6486ffd997-2rvsr\" (UID: \"f36dea70-20ce-4185-ac0d-72469bed6792\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:24:37.092366 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.092341 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2rvsr"] Jul 05 13:24:37.156111 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.156088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bnn\" (UniqueName: \"kubernetes.io/projected/f36dea70-20ce-4185-ac0d-72469bed6792-kube-api-access-z2bnn\") pod \"cert-manager-webhook-6486ffd997-2rvsr\" (UID: \"f36dea70-20ce-4185-ac0d-72469bed6792\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:24:37.156217 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.156141 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36dea70-20ce-4185-ac0d-72469bed6792-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2rvsr\" (UID: \"f36dea70-20ce-4185-ac0d-72469bed6792\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:24:37.195432 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.195410 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36dea70-20ce-4185-ac0d-72469bed6792-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2rvsr\" (UID: \"f36dea70-20ce-4185-ac0d-72469bed6792\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:24:37.200000 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.199970 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2bnn\" (UniqueName: \"kubernetes.io/projected/f36dea70-20ce-4185-ac0d-72469bed6792-kube-api-access-z2bnn\") pod \"cert-manager-webhook-6486ffd997-2rvsr\" (UID: \"f36dea70-20ce-4185-ac0d-72469bed6792\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:24:37.308499 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.308462 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:24:37.446000 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.445953 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2rvsr"] Jul 05 13:24:37.449258 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:24:37.449223 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf36dea70_20ce_4185_ac0d_72469bed6792.slice/crio-65594908f74310d86c9263c10e3aed037224391da00ffaa34bd790567043040b WatchSource:0}: Error finding container 65594908f74310d86c9263c10e3aed037224391da00ffaa34bd790567043040b: Status 404 returned error can't find the container with id 65594908f74310d86c9263c10e3aed037224391da00ffaa34bd790567043040b Jul 05 13:24:37.451185 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.451170 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:24:37.500054 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:37.500023 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" event={"ID":"f36dea70-20ce-4185-ac0d-72469bed6792","Type":"ContainerStarted","Data":"65594908f74310d86c9263c10e3aed037224391da00ffaa34bd790567043040b"} Jul 05 13:24:40.515992 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:40.515945 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" event={"ID":"f36dea70-20ce-4185-ac0d-72469bed6792","Type":"ContainerStarted","Data":"da6e520fa944ea5b99e297e5b88df367e948b6c19e55efe848fc6b4f938e139a"} Jul 05 13:24:40.516404 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:40.516082 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:24:40.533283 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:40.533202 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" podStartSLOduration=1.713882056 podStartE2EDuration="4.533187318s" totalImagesPullingTime="2.819305262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:24:40.531987865 +0000 UTC m=+385.160353618" watchObservedRunningTime="2026-07-05 13:24:40.533187318 +0000 UTC m=+385.161553072" Jul 05 13:24:46.520932 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:24:46.520898 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2rvsr" Jul 05 13:25:09.867064 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:09.867028 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc"] Jul 05 13:25:09.870376 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:09.870359 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:09.883656 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:09.883637 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 05 13:25:09.883862 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:09.883844 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 05 13:25:09.888436 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:09.888419 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 05 13:25:09.889056 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:09.889038 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 05 13:25:09.889160 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:09.889079 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-24rg9\"" Jul 05 13:25:09.901230 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:09.901210 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc"] Jul 05 13:25:10.013084 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.013059 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7217a8a5-d941-4a4d-8e55-bedac65d519a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.013200 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.013108 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7217a8a5-d941-4a4d-8e55-bedac65d519a-webhook-cert\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.013200 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.013136 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7217a8a5-d941-4a4d-8e55-bedac65d519a-tmp\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.013277 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.013205 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qfs7\" (UniqueName: \"kubernetes.io/projected/7217a8a5-d941-4a4d-8e55-bedac65d519a-kube-api-access-5qfs7\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.013277 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.013247 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7217a8a5-d941-4a4d-8e55-bedac65d519a-manifests\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.114462 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.114439 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7217a8a5-d941-4a4d-8e55-bedac65d519a-webhook-cert\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.114570 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.114468 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7217a8a5-d941-4a4d-8e55-bedac65d519a-tmp\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.114570 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.114500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfs7\" (UniqueName: \"kubernetes.io/projected/7217a8a5-d941-4a4d-8e55-bedac65d519a-kube-api-access-5qfs7\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.114570 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.114543 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7217a8a5-d941-4a4d-8e55-bedac65d519a-manifests\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.114718 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.114574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7217a8a5-d941-4a4d-8e55-bedac65d519a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.115012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.114982 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7217a8a5-d941-4a4d-8e55-bedac65d519a-manifests\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.116984 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.116966 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7217a8a5-d941-4a4d-8e55-bedac65d519a-tmp\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.117248 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.117201 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7217a8a5-d941-4a4d-8e55-bedac65d519a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.117248 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.117234 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7217a8a5-d941-4a4d-8e55-bedac65d519a-webhook-cert\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.139156 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.139129 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qfs7\" (UniqueName: \"kubernetes.io/projected/7217a8a5-d941-4a4d-8e55-bedac65d519a-kube-api-access-5qfs7\") pod \"opendatahub-operator-controller-manager-54547f4bcb-qhsrc\" (UID: \"7217a8a5-d941-4a4d-8e55-bedac65d519a\") " pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.180073 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.180055 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:10.345286 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.345260 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc"] Jul 05 13:25:10.346809 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:25:10.346782 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7217a8a5_d941_4a4d_8e55_bedac65d519a.slice/crio-2646179ffbbb517a5be1b48274a497a9808a3d3443f962e2596c75c641b652b3 WatchSource:0}: Error finding container 2646179ffbbb517a5be1b48274a497a9808a3d3443f962e2596c75c641b652b3: Status 404 returned error can't find the container with id 2646179ffbbb517a5be1b48274a497a9808a3d3443f962e2596c75c641b652b3 Jul 05 13:25:10.629696 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:10.629646 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" event={"ID":"7217a8a5-d941-4a4d-8e55-bedac65d519a","Type":"ContainerStarted","Data":"2646179ffbbb517a5be1b48274a497a9808a3d3443f962e2596c75c641b652b3"} Jul 05 13:25:13.647000 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:13.646967 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" event={"ID":"7217a8a5-d941-4a4d-8e55-bedac65d519a","Type":"ContainerStarted","Data":"7cffacd2ab43905c3a26198efb81660dba1b16db5f91ddc6bcf30578359da406"} Jul 05 13:25:14.653013 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:14.652978 2579 generic.go:356] "Generic (PLEG): container finished" podID="7217a8a5-d941-4a4d-8e55-bedac65d519a" containerID="7cffacd2ab43905c3a26198efb81660dba1b16db5f91ddc6bcf30578359da406" exitCode=0 Jul 05 13:25:14.653378 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:14.653060 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" event={"ID":"7217a8a5-d941-4a4d-8e55-bedac65d519a","Type":"ContainerDied","Data":"7cffacd2ab43905c3a26198efb81660dba1b16db5f91ddc6bcf30578359da406"} Jul 05 13:25:15.661328 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:15.661291 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" event={"ID":"7217a8a5-d941-4a4d-8e55-bedac65d519a","Type":"ContainerStarted","Data":"4bc3ddcf928e2f8e6622f5cc67fca0393cc099193d1c99223adf8fbe4ec8a133"} Jul 05 13:25:15.661741 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:15.661378 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:15.691918 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:15.691873 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" podStartSLOduration=2.575940843 podStartE2EDuration="6.691857443s" totalImagesPullingTime="3.025016897s" totalInitContainerRuntime="1.090899703s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:15.690408105 +0000 UTC m=+420.318774021" watchObservedRunningTime="2026-07-05 13:25:15.691857443 +0000 UTC m=+420.320223198" Jul 05 13:25:26.667894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:26.667864 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54547f4bcb-qhsrc" Jul 05 13:25:36.810323 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:36.810234 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd"] Jul 05 13:25:36.924051 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:36.924016 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd"] Jul 05 13:25:36.924237 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:36.924114 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:36.931205 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:36.931172 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 05 13:25:36.931341 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:36.931227 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 05 13:25:36.931341 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:36.931268 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 05 13:25:36.931470 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:36.931274 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 05 13:25:36.931470 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:36.931227 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 05 13:25:36.931662 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:36.931644 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-djcd4\"" Jul 05 13:25:37.030776 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.030752 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95ab55db-db4f-4c88-90df-3840cc7d5727-cert\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.030776 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.030784 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vh4c\" (UniqueName: \"kubernetes.io/projected/95ab55db-db4f-4c88-90df-3840cc7d5727-kube-api-access-9vh4c\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.030965 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.030835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/95ab55db-db4f-4c88-90df-3840cc7d5727-manager-config\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.030965 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.030889 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/95ab55db-db4f-4c88-90df-3840cc7d5727-metrics-cert\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.131238 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.131180 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vh4c\" (UniqueName: \"kubernetes.io/projected/95ab55db-db4f-4c88-90df-3840cc7d5727-kube-api-access-9vh4c\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.131238 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.131218 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/95ab55db-db4f-4c88-90df-3840cc7d5727-manager-config\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.131383 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.131238 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/95ab55db-db4f-4c88-90df-3840cc7d5727-metrics-cert\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.131383 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.131289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95ab55db-db4f-4c88-90df-3840cc7d5727-cert\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.131815 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.131796 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/95ab55db-db4f-4c88-90df-3840cc7d5727-manager-config\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.133870 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.133852 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95ab55db-db4f-4c88-90df-3840cc7d5727-cert\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.134037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.134014 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/95ab55db-db4f-4c88-90df-3840cc7d5727-metrics-cert\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.167966 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.167940 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vh4c\" (UniqueName: \"kubernetes.io/projected/95ab55db-db4f-4c88-90df-3840cc7d5727-kube-api-access-9vh4c\") pod \"lws-controller-manager-cfb4f694f-6hvsd\" (UID: \"95ab55db-db4f-4c88-90df-3840cc7d5727\") " pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.233265 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.233233 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:37.412718 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.412669 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd"] Jul 05 13:25:37.415062 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:25:37.415035 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95ab55db_db4f_4c88_90df_3840cc7d5727.slice/crio-8752e798ccd6b52852cb9312e73d5f2ef277b2df9be11aa91bcca8a8ff1d3745 WatchSource:0}: Error finding container 8752e798ccd6b52852cb9312e73d5f2ef277b2df9be11aa91bcca8a8ff1d3745: Status 404 returned error can't find the container with id 8752e798ccd6b52852cb9312e73d5f2ef277b2df9be11aa91bcca8a8ff1d3745 Jul 05 13:25:37.748351 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:37.748264 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" event={"ID":"95ab55db-db4f-4c88-90df-3840cc7d5727","Type":"ContainerStarted","Data":"8752e798ccd6b52852cb9312e73d5f2ef277b2df9be11aa91bcca8a8ff1d3745"} Jul 05 13:25:40.762262 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:40.762226 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" event={"ID":"95ab55db-db4f-4c88-90df-3840cc7d5727","Type":"ContainerStarted","Data":"2edd6189944f9e8a8ba423c0dbd11736b6c7a3263cc5450da182275a30344ca5"} Jul 05 13:25:40.762663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:40.762360 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:40.912831 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:40.912783 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" podStartSLOduration=2.505522896 podStartE2EDuration="4.912767934s" totalImagesPullingTime="2.407245038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:40.911446417 +0000 UTC m=+445.539812171" watchObservedRunningTime="2026-07-05 13:25:40.912767934 +0000 UTC m=+445.541133689" Jul 05 13:25:41.636240 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.636210 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9"] Jul 05 13:25:41.639480 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.639464 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.646911 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.646895 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 05 13:25:41.647500 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.647485 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 05 13:25:41.647548 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.647506 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zmxk7\"" Jul 05 13:25:41.652222 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.652207 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 05 13:25:41.679908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.679879 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9"] Jul 05 13:25:41.768646 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.768616 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f91e152d-e714-45f0-9cd3-fdad49637936-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.769060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.768654 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.769060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.768712 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.769060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.768765 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.769060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.768793 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f91e152d-e714-45f0-9cd3-fdad49637936-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.769060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.768835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.769060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.768862 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px9c2\" (UniqueName: \"kubernetes.io/projected/f91e152d-e714-45f0-9cd3-fdad49637936-kube-api-access-px9c2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.769060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.768928 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f91e152d-e714-45f0-9cd3-fdad49637936-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.769060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.768962 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870280 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870253 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870462 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870462 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870334 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870462 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870349 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f91e152d-e714-45f0-9cd3-fdad49637936-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870462 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870375 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870462 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870415 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px9c2\" (UniqueName: \"kubernetes.io/projected/f91e152d-e714-45f0-9cd3-fdad49637936-kube-api-access-px9c2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870462 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870452 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f91e152d-e714-45f0-9cd3-fdad49637936-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870828 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870483 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870828 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870781 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870931 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870829 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870931 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f91e152d-e714-45f0-9cd3-fdad49637936-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.870931 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.870882 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.871405 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.871340 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f91e152d-e714-45f0-9cd3-fdad49637936-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.871405 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.871349 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.872744 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.872723 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f91e152d-e714-45f0-9cd3-fdad49637936-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.873161 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.873145 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f91e152d-e714-45f0-9cd3-fdad49637936-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.910622 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.910562 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f91e152d-e714-45f0-9cd3-fdad49637936-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.910733 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.910658 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px9c2\" (UniqueName: \"kubernetes.io/projected/f91e152d-e714-45f0-9cd3-fdad49637936-kube-api-access-px9c2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dgls9\" (UID: \"f91e152d-e714-45f0-9cd3-fdad49637936\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:41.950451 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:41.950429 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:42.129396 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:25:42.129340 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf91e152d_e714_45f0_9cd3_fdad49637936.slice/crio-156951332dd7d89980d556d082b9e3087657557c845bfaf41a473ca67fa417a6 WatchSource:0}: Error finding container 156951332dd7d89980d556d082b9e3087657557c845bfaf41a473ca67fa417a6: Status 404 returned error can't find the container with id 156951332dd7d89980d556d082b9e3087657557c845bfaf41a473ca67fa417a6 Jul 05 13:25:42.131905 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:42.131881 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9"] Jul 05 13:25:42.773359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:42.773325 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" event={"ID":"f91e152d-e714-45f0-9cd3-fdad49637936","Type":"ContainerStarted","Data":"156951332dd7d89980d556d082b9e3087657557c845bfaf41a473ca67fa417a6"} Jul 05 13:25:44.446015 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:44.445971 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:44.446347 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:44.446049 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:44.446347 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:44.446092 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:44.783667 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:44.783572 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" event={"ID":"f91e152d-e714-45f0-9cd3-fdad49637936","Type":"ContainerStarted","Data":"ab0362924378189177936c6dffebb71f461b525cec7d2ed5dec436b735aec402"} Jul 05 13:25:44.910214 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:44.910163 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" podStartSLOduration=1.59627463 podStartE2EDuration="3.910146816s" totalImagesPullingTime="2.313872186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:44.910000983 +0000 UTC m=+449.538366737" watchObservedRunningTime="2026-07-05 13:25:44.910146816 +0000 UTC m=+449.538512572" Jul 05 13:25:44.951041 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:44.951008 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:45.955609 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:45.955581 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:46.791421 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:46.791391 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:46.792236 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:46.792216 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dgls9" Jul 05 13:25:48.762192 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.762158 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4"] Jul 05 13:25:48.765583 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.765565 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.779617 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.779598 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ndwlb\"" Jul 05 13:25:48.779895 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.779879 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 05 13:25:48.779944 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.779914 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 05 13:25:48.780311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.780300 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 05 13:25:48.828186 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.828151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.828186 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.828185 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.828358 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.828245 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.828398 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.828364 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbe8e3cf-3126-4cad-880c-313c51461d20-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.828435 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.828393 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.828474 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.828439 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dbe8e3cf-3126-4cad-880c-313c51461d20-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.828509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.828476 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj86p\" (UniqueName: \"kubernetes.io/projected/dbe8e3cf-3126-4cad-880c-313c51461d20-kube-api-access-dj86p\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.880187 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.880154 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4"] Jul 05 13:25:48.929952 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.929923 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.930137 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.929963 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.930137 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.930005 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.930137 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.930042 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbe8e3cf-3126-4cad-880c-313c51461d20-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.930137 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.930068 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.930137 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.930107 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dbe8e3cf-3126-4cad-880c-313c51461d20-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.930137 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.930132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj86p\" (UniqueName: \"kubernetes.io/projected/dbe8e3cf-3126-4cad-880c-313c51461d20-kube-api-access-dj86p\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.930917 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.930840 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.932488 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.932463 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbe8e3cf-3126-4cad-880c-313c51461d20-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.932610 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.932596 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dbe8e3cf-3126-4cad-880c-313c51461d20-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.932996 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.932974 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.933387 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.933370 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.965745 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.965713 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj86p\" (UniqueName: \"kubernetes.io/projected/dbe8e3cf-3126-4cad-880c-313c51461d20-kube-api-access-dj86p\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:48.982109 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:48.982070 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbe8e3cf-3126-4cad-880c-313c51461d20-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-69mw4\" (UID: \"dbe8e3cf-3126-4cad-880c-313c51461d20\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:49.075479 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:49.075447 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:49.491551 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:49.491471 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4"] Jul 05 13:25:49.500808 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:25:49.500783 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbe8e3cf_3126_4cad_880c_313c51461d20.slice/crio-cee588976ac7386158279b9fa88c98102f506acb4ab26d3f9582c61c5342428d WatchSource:0}: Error finding container cee588976ac7386158279b9fa88c98102f506acb4ab26d3f9582c61c5342428d: Status 404 returned error can't find the container with id cee588976ac7386158279b9fa88c98102f506acb4ab26d3f9582c61c5342428d Jul 05 13:25:49.805925 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:49.805893 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" event={"ID":"dbe8e3cf-3126-4cad-880c-313c51461d20","Type":"ContainerStarted","Data":"cee588976ac7386158279b9fa88c98102f506acb4ab26d3f9582c61c5342428d"} Jul 05 13:25:51.577639 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:51.577606 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:51.577983 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:51.577665 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:25:51.768121 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:51.768085 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-cfb4f694f-6hvsd" Jul 05 13:25:51.816493 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:51.816459 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" event={"ID":"dbe8e3cf-3126-4cad-880c-313c51461d20","Type":"ContainerStarted","Data":"4c079ae9f80e6f7d59a08ec270f07298536da2ba7a933cb833585f3a2333dee1"} Jul 05 13:25:51.816648 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:51.816608 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:52.822776 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:52.822743 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-69mw4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 05 13:25:52.823136 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:52.822804 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" podUID="dbe8e3cf-3126-4cad-880c-313c51461d20" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 13:25:55.822507 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:55.822478 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" Jul 05 13:25:56.128162 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:25:56.128062 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-69mw4" podStartSLOduration=6.053620421 podStartE2EDuration="8.128044541s" totalImagesPullingTime="2.07442412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:25:52.225663926 +0000 UTC m=+456.854029681" watchObservedRunningTime="2026-07-05 13:25:56.128044541 +0000 UTC m=+460.756410295" Jul 05 13:26:47.130889 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.130856 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7446c94d6f-rshlr"] Jul 05 13:26:47.134690 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.134663 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.170382 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.170354 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7446c94d6f-rshlr"] Jul 05 13:26:47.224666 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.224638 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rscrs\" (UniqueName: \"kubernetes.io/projected/16249bc8-bf8f-4389-895c-5ea504981578-kube-api-access-rscrs\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.224814 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.224698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-console-config\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.224814 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.224726 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-trusted-ca-bundle\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.224814 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.224763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-service-ca\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.224814 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.224782 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-oauth-serving-cert\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.224814 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.224808 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16249bc8-bf8f-4389-895c-5ea504981578-console-serving-cert\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.224985 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.224825 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16249bc8-bf8f-4389-895c-5ea504981578-console-oauth-config\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.325866 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.325837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16249bc8-bf8f-4389-895c-5ea504981578-console-serving-cert\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.325866 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.325867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16249bc8-bf8f-4389-895c-5ea504981578-console-oauth-config\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.326108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.325933 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rscrs\" (UniqueName: \"kubernetes.io/projected/16249bc8-bf8f-4389-895c-5ea504981578-kube-api-access-rscrs\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.326108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.325953 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-console-config\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.326108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.325970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-trusted-ca-bundle\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.326108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.326012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-service-ca\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.326307 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.326146 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-oauth-serving-cert\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.326846 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.326823 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-oauth-serving-cert\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.326965 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.326905 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-trusted-ca-bundle\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.327012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.326970 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-console-config\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.327012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.326997 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16249bc8-bf8f-4389-895c-5ea504981578-service-ca\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.328596 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.328572 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16249bc8-bf8f-4389-895c-5ea504981578-console-serving-cert\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.328740 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.328601 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16249bc8-bf8f-4389-895c-5ea504981578-console-oauth-config\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.339293 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.339276 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rscrs\" (UniqueName: \"kubernetes.io/projected/16249bc8-bf8f-4389-895c-5ea504981578-kube-api-access-rscrs\") pod \"console-7446c94d6f-rshlr\" (UID: \"16249bc8-bf8f-4389-895c-5ea504981578\") " pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.443774 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.443713 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:47.591692 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:47.591647 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7446c94d6f-rshlr"] Jul 05 13:26:47.594505 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:26:47.594476 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16249bc8_bf8f_4389_895c_5ea504981578.slice/crio-c27d3dc13fa0a394e4962476faabec288bcc1624cf59f2c88208219a31fdbdc4 WatchSource:0}: Error finding container c27d3dc13fa0a394e4962476faabec288bcc1624cf59f2c88208219a31fdbdc4: Status 404 returned error can't find the container with id c27d3dc13fa0a394e4962476faabec288bcc1624cf59f2c88208219a31fdbdc4 Jul 05 13:26:48.048613 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:48.048584 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7446c94d6f-rshlr" event={"ID":"16249bc8-bf8f-4389-895c-5ea504981578","Type":"ContainerStarted","Data":"ed66b1085a49e65081d7c4ac1efb73ca96dfd8c5e087e1b542406a047afaa2f7"} Jul 05 13:26:48.048812 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:48.048630 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7446c94d6f-rshlr" event={"ID":"16249bc8-bf8f-4389-895c-5ea504981578","Type":"ContainerStarted","Data":"c27d3dc13fa0a394e4962476faabec288bcc1624cf59f2c88208219a31fdbdc4"} Jul 05 13:26:48.089195 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:48.089152 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7446c94d6f-rshlr" podStartSLOduration=1.089134269 podStartE2EDuration="1.089134269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:26:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:26:48.087895314 +0000 UTC m=+512.716261094" watchObservedRunningTime="2026-07-05 13:26:48.089134269 +0000 UTC m=+512.717500085" Jul 05 13:26:57.444806 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:57.444767 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:57.445247 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:57.444835 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:57.449196 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:57.449174 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:58.093295 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:58.093268 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7446c94d6f-rshlr" Jul 05 13:26:58.191025 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:26:58.190995 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57fdb5d6bd-njwbs"] Jul 05 13:27:23.210604 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.210524 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57fdb5d6bd-njwbs" podUID="b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" containerName="console" containerID="cri-o://bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861" gracePeriod=15 Jul 05 13:27:23.461050 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.460989 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:27:23.640562 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.640529 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-service-ca\") pod \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " Jul 05 13:27:23.640768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.640587 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-oauth-serving-cert\") pod \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " Jul 05 13:27:23.640768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.640619 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-trusted-ca-bundle\") pod \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " Jul 05 13:27:23.640768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.640652 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvrqw\" (UniqueName: \"kubernetes.io/projected/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-kube-api-access-nvrqw\") pod \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " Jul 05 13:27:23.640768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.640674 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-config\") pod \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " Jul 05 13:27:23.640974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.640839 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-serving-cert\") pod \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " Jul 05 13:27:23.640974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.640883 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-oauth-config\") pod \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\" (UID: \"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae\") " Jul 05 13:27:23.641084 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.641059 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-service-ca" (OuterVolumeSpecName: "service-ca") pod "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" (UID: "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:27:23.641156 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.641064 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" (UID: "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:27:23.641213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.641183 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-config" (OuterVolumeSpecName: "console-config") pod "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" (UID: "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:27:23.641213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.641191 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" (UID: "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:27:23.641292 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.641278 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-service-ca\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:27:23.641328 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.641298 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-oauth-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:27:23.643110 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.643082 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" (UID: "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:27:23.643205 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.643141 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" (UID: "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 13:27:23.643205 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.643147 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-kube-api-access-nvrqw" (OuterVolumeSpecName: "kube-api-access-nvrqw") pod "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" (UID: "b9b5273f-ddbc-423c-b5f3-c31b1bc248ae"). InnerVolumeSpecName "kube-api-access-nvrqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:27:23.741768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.741710 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-trusted-ca-bundle\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:27:23.741768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.741731 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvrqw\" (UniqueName: \"kubernetes.io/projected/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-kube-api-access-nvrqw\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:27:23.741768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.741744 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:27:23.741768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.741754 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-serving-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:27:23.741768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:23.741765 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae-console-oauth-config\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:27:24.203856 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:24.203823 2579 generic.go:356] "Generic (PLEG): container finished" podID="b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" containerID="bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861" exitCode=0 Jul 05 13:27:24.204033 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:24.203874 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fdb5d6bd-njwbs" Jul 05 13:27:24.204033 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:24.203905 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fdb5d6bd-njwbs" event={"ID":"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae","Type":"ContainerDied","Data":"bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861"} Jul 05 13:27:24.204033 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:24.203955 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fdb5d6bd-njwbs" event={"ID":"b9b5273f-ddbc-423c-b5f3-c31b1bc248ae","Type":"ContainerDied","Data":"a1d7556edc18012867baa7f146bb9ffafa7cd41da71aad732c6e87de927ad257"} Jul 05 13:27:24.204033 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:24.203984 2579 scope.go:117] "RemoveContainer" containerID="bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861" Jul 05 13:27:24.212697 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:24.212513 2579 scope.go:117] "RemoveContainer" containerID="bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861" Jul 05 13:27:24.212914 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:27:24.212812 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861\": container with ID starting with bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861 not found: ID does not exist" containerID="bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861" Jul 05 13:27:24.212914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:24.212835 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861"} err="failed to get container status \"bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861\": rpc error: code = NotFound desc = could not find container \"bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861\": container with ID starting with bf760048b4e2bd82bb59c1030371ebf68abb0c13e21c58d73d1ba2167560a861 not found: ID does not exist" Jul 05 13:27:24.263219 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:24.263195 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57fdb5d6bd-njwbs"] Jul 05 13:27:24.273080 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:24.273056 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57fdb5d6bd-njwbs"] Jul 05 13:27:25.892987 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:27:25.892954 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" path="/var/lib/kubelet/pods/b9b5273f-ddbc-423c-b5f3-c31b1bc248ae/volumes" Jul 05 13:28:04.635977 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.635947 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h"] Jul 05 13:28:04.636526 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.636400 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" containerName="console" Jul 05 13:28:04.636526 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.636419 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" containerName="console" Jul 05 13:28:04.636711 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.636694 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9b5273f-ddbc-423c-b5f3-c31b1bc248ae" containerName="console" Jul 05 13:28:04.639916 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.639895 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.642392 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.642372 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-jkvdb\"" Jul 05 13:28:04.652403 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.652381 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h"] Jul 05 13:28:04.767849 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.767818 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmtlv\" (UniqueName: \"kubernetes.io/projected/490ff508-add3-40b5-bc75-6a3f69acd8c6-kube-api-access-vmtlv\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.767849 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.767852 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.768069 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.767877 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.768069 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.767897 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.768069 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.767970 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/490ff508-add3-40b5-bc75-6a3f69acd8c6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.768069 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.768013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.768069 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.768036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.768069 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.768058 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.768285 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.768097 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.868831 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.868801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/490ff508-add3-40b5-bc75-6a3f69acd8c6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.868986 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.868837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.868986 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.868863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.868986 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.868885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869164 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869020 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869164 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869061 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmtlv\" (UniqueName: \"kubernetes.io/projected/490ff508-add3-40b5-bc75-6a3f69acd8c6-kube-api-access-vmtlv\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869164 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869164 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869130 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869164 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869162 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869430 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869313 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869497 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869472 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869555 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869504 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869555 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869526 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.869662 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.869606 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/490ff508-add3-40b5-bc75-6a3f69acd8c6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.871316 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.871294 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.871799 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.871773 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.877951 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.877922 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/490ff508-add3-40b5-bc75-6a3f69acd8c6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.878053 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.877997 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmtlv\" (UniqueName: \"kubernetes.io/projected/490ff508-add3-40b5-bc75-6a3f69acd8c6-kube-api-access-vmtlv\") pod \"maas-default-gateway-openshift-default-56654dcf7-fzz4h\" (UID: \"490ff508-add3-40b5-bc75-6a3f69acd8c6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:04.949704 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:04.949620 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:05.082755 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:05.082725 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h"] Jul 05 13:28:05.085707 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:28:05.085660 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490ff508_add3_40b5_bc75_6a3f69acd8c6.slice/crio-12a2efea0241abb4db5e9feb15bf89123094858dc4b8b48aa19e19f4a1d6fb9a WatchSource:0}: Error finding container 12a2efea0241abb4db5e9feb15bf89123094858dc4b8b48aa19e19f4a1d6fb9a: Status 404 returned error can't find the container with id 12a2efea0241abb4db5e9feb15bf89123094858dc4b8b48aa19e19f4a1d6fb9a Jul 05 13:28:05.087916 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:05.087885 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:28:05.088019 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:05.087966 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:28:05.088019 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:05.088011 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:28:05.388046 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:05.388011 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" event={"ID":"490ff508-add3-40b5-bc75-6a3f69acd8c6","Type":"ContainerStarted","Data":"66eaee5867894d67fb801ba676c1f9293887e3d2005f639dda9e7f53a400a085"} Jul 05 13:28:05.388232 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:05.388069 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" event={"ID":"490ff508-add3-40b5-bc75-6a3f69acd8c6","Type":"ContainerStarted","Data":"12a2efea0241abb4db5e9feb15bf89123094858dc4b8b48aa19e19f4a1d6fb9a"} Jul 05 13:28:05.408072 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:05.408029 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" podStartSLOduration=1.408013888 podStartE2EDuration="1.408013888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:28:05.406633997 +0000 UTC m=+590.034999753" watchObservedRunningTime="2026-07-05 13:28:05.408013888 +0000 UTC m=+590.036379710" Jul 05 13:28:05.950593 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:05.950567 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:06.955734 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:06.955704 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:07.396824 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:07.396794 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:07.397755 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:07.397735 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:28:27.995372 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:27.995341 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-pljv9"] Jul 05 13:28:27.999450 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:27.999427 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:28:28.001836 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.001813 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ms8zg\"" Jul 05 13:28:28.001941 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.001854 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 05 13:28:28.009605 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.009587 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-pljv9"] Jul 05 13:28:28.060131 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.060102 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg45x\" (UniqueName: \"kubernetes.io/projected/6317a779-54ed-46cc-94bd-77a561f3a5f8-kube-api-access-fg45x\") pod \"postgres-69fb4594cf-pljv9\" (UID: \"6317a779-54ed-46cc-94bd-77a561f3a5f8\") " pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:28:28.060238 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.060176 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6317a779-54ed-46cc-94bd-77a561f3a5f8-data\") pod \"postgres-69fb4594cf-pljv9\" (UID: \"6317a779-54ed-46cc-94bd-77a561f3a5f8\") " pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:28:28.161534 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.161503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6317a779-54ed-46cc-94bd-77a561f3a5f8-data\") pod \"postgres-69fb4594cf-pljv9\" (UID: \"6317a779-54ed-46cc-94bd-77a561f3a5f8\") " pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:28:28.161724 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.161553 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg45x\" (UniqueName: \"kubernetes.io/projected/6317a779-54ed-46cc-94bd-77a561f3a5f8-kube-api-access-fg45x\") pod \"postgres-69fb4594cf-pljv9\" (UID: \"6317a779-54ed-46cc-94bd-77a561f3a5f8\") " pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:28:28.161868 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.161848 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6317a779-54ed-46cc-94bd-77a561f3a5f8-data\") pod \"postgres-69fb4594cf-pljv9\" (UID: \"6317a779-54ed-46cc-94bd-77a561f3a5f8\") " pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:28:28.170125 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.170102 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg45x\" (UniqueName: \"kubernetes.io/projected/6317a779-54ed-46cc-94bd-77a561f3a5f8-kube-api-access-fg45x\") pod \"postgres-69fb4594cf-pljv9\" (UID: \"6317a779-54ed-46cc-94bd-77a561f3a5f8\") " pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:28:28.310882 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.310859 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:28:28.438159 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.438135 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-pljv9"] Jul 05 13:28:28.440124 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:28:28.440100 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6317a779_54ed_46cc_94bd_77a561f3a5f8.slice/crio-5c56119a181e664bee4709fa48f512444482e57fc558e8130d4b82e9904fe08d WatchSource:0}: Error finding container 5c56119a181e664bee4709fa48f512444482e57fc558e8130d4b82e9904fe08d: Status 404 returned error can't find the container with id 5c56119a181e664bee4709fa48f512444482e57fc558e8130d4b82e9904fe08d Jul 05 13:28:28.496604 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:28.496581 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-pljv9" event={"ID":"6317a779-54ed-46cc-94bd-77a561f3a5f8","Type":"ContainerStarted","Data":"5c56119a181e664bee4709fa48f512444482e57fc558e8130d4b82e9904fe08d"} Jul 05 13:28:33.525895 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:33.525844 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-pljv9" event={"ID":"6317a779-54ed-46cc-94bd-77a561f3a5f8","Type":"ContainerStarted","Data":"f4e87c621666dd6d46cfe5c0b59707a35c8ad9b5141518f3a94699f95e18d23c"} Jul 05 13:28:33.526278 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:33.525934 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:28:33.541635 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:33.541551 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-pljv9" podStartSLOduration=2.009325153 podStartE2EDuration="6.541529672s" totalImagesPullingTime="4.532204519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:28:33.540929049 +0000 UTC m=+618.169294805" watchObservedRunningTime="2026-07-05 13:28:33.541529672 +0000 UTC m=+618.169895430" Jul 05 13:28:39.562989 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:28:39.562921 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-pljv9" Jul 05 13:29:32.794497 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:32.794465 2579 generic.go:356] "Generic (PLEG): container finished" podID="490ff508-add3-40b5-bc75-6a3f69acd8c6" containerID="66eaee5867894d67fb801ba676c1f9293887e3d2005f639dda9e7f53a400a085" exitCode=0 Jul 05 13:29:32.794864 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:32.794540 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" event={"ID":"490ff508-add3-40b5-bc75-6a3f69acd8c6","Type":"ContainerDied","Data":"66eaee5867894d67fb801ba676c1f9293887e3d2005f639dda9e7f53a400a085"} Jul 05 13:29:32.794992 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:32.794978 2579 scope.go:117] "RemoveContainer" containerID="66eaee5867894d67fb801ba676c1f9293887e3d2005f639dda9e7f53a400a085" Jul 05 13:29:32.796009 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:32.795984 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:29:32.796106 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:32.796074 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:29:32.796164 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:32.796122 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:29:33.803730 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:33.803660 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" event={"ID":"490ff508-add3-40b5-bc75-6a3f69acd8c6","Type":"ContainerStarted","Data":"b6860d69948e7140c38a86f41a89b429ebac2cf3d1a1f7be472c96c6c6e83846"} Jul 05 13:29:33.950686 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:33.950641 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:29:33.951742 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:33.951712 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fzz4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 05 13:29:33.951868 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:33.951775 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" podUID="490ff508-add3-40b5-bc75-6a3f69acd8c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 05 13:29:34.950118 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:34.950086 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fzz4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 05 13:29:34.950506 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:34.950145 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" podUID="490ff508-add3-40b5-bc75-6a3f69acd8c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 05 13:29:35.950413 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:35.950373 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fzz4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 05 13:29:35.950916 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:35.950432 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" podUID="490ff508-add3-40b5-bc75-6a3f69acd8c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 05 13:29:36.950603 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:36.950570 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-fzz4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 05 13:29:36.950996 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:36.950631 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" podUID="490ff508-add3-40b5-bc75-6a3f69acd8c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 05 13:29:37.397704 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:37.397643 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:29:37.954231 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:37.954200 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:29:37.955113 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:29:37.955093 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-fzz4h" Jul 05 13:30:28.923123 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.923053 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7"] Jul 05 13:30:28.926817 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.926801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:28.930538 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.930515 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 05 13:30:28.930663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.930547 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bmbgn\"" Jul 05 13:30:28.930663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.930546 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 05 13:30:28.930663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.930598 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 05 13:30:28.954751 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.954732 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7"] Jul 05 13:30:28.978849 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.978822 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:28.978948 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.978862 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blpvh\" (UniqueName: \"kubernetes.io/projected/5dc85d56-9c91-42f2-9048-708098ed16cc-kube-api-access-blpvh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:28.978948 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.978885 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:28.979039 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.978953 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:28.979039 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.978983 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:28.979039 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:28.978999 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dc85d56-9c91-42f2-9048-708098ed16cc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.080329 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.080301 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.080446 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.080347 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blpvh\" (UniqueName: \"kubernetes.io/projected/5dc85d56-9c91-42f2-9048-708098ed16cc-kube-api-access-blpvh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.080446 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.080378 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.080446 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.080435 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.080607 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.080475 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.080607 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.080503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dc85d56-9c91-42f2-9048-708098ed16cc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.080779 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.080757 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.080843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.080780 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.080843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.080834 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.082637 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.082619 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dc85d56-9c91-42f2-9048-708098ed16cc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.082992 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.082972 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dc85d56-9c91-42f2-9048-708098ed16cc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.089382 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.089358 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blpvh\" (UniqueName: \"kubernetes.io/projected/5dc85d56-9c91-42f2-9048-708098ed16cc-kube-api-access-blpvh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xvlt7\" (UID: \"5dc85d56-9c91-42f2-9048-708098ed16cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.237419 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.237338 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:29.366477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.366454 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7"] Jul 05 13:30:29.368629 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:30:29.368604 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc85d56_9c91_42f2_9048_708098ed16cc.slice/crio-74688112f7b83e8cb2635dcf5c2385164c7f650ca5e1df95462499733adf0312 WatchSource:0}: Error finding container 74688112f7b83e8cb2635dcf5c2385164c7f650ca5e1df95462499733adf0312: Status 404 returned error can't find the container with id 74688112f7b83e8cb2635dcf5c2385164c7f650ca5e1df95462499733adf0312 Jul 05 13:30:29.370085 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:29.370040 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:30:30.044418 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:30.044380 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" event={"ID":"5dc85d56-9c91-42f2-9048-708098ed16cc","Type":"ContainerStarted","Data":"74688112f7b83e8cb2635dcf5c2385164c7f650ca5e1df95462499733adf0312"} Jul 05 13:30:31.050336 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:31.050300 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" event={"ID":"5dc85d56-9c91-42f2-9048-708098ed16cc","Type":"ContainerStarted","Data":"a62f786f30feb2a98ecfd4678dc8cbf328708c4cface05e2675429d3c632307c"} Jul 05 13:30:31.070806 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:31.070753 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" podStartSLOduration=2.094569713 podStartE2EDuration="3.070735336s" totalImagesPullingTime="976.165623ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:30:31.069737616 +0000 UTC m=+735.698103370" watchObservedRunningTime="2026-07-05 13:30:31.070735336 +0000 UTC m=+735.699101090" Jul 05 13:30:39.237983 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:39.237953 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:39.238380 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:39.238003 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:39.250482 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:39.250458 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:40.100366 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:40.100335 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xvlt7" Jul 05 13:30:53.808311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:53.808282 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56"] Jul 05 13:30:53.811455 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:53.811433 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:53.814105 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:53.814081 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 05 13:30:53.823530 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:53.823511 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56"] Jul 05 13:30:53.999916 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:53.999885 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.000072 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:53.999918 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4vqn\" (UniqueName: \"kubernetes.io/projected/4e3dfe9b-104d-4a61-9766-205643d58666-kube-api-access-c4vqn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.000072 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:53.999988 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.000072 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.000032 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.000072 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.000056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.000225 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.000073 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3dfe9b-104d-4a61-9766-205643d58666-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.100971 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.100906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.100971 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.100935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4vqn\" (UniqueName: \"kubernetes.io/projected/4e3dfe9b-104d-4a61-9766-205643d58666-kube-api-access-c4vqn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.100971 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.100965 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.101194 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.100993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.101194 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.101022 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.101194 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.101047 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3dfe9b-104d-4a61-9766-205643d58666-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.101381 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.101363 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.101443 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.101424 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.101509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.101490 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.103392 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.103361 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3dfe9b-104d-4a61-9766-205643d58666-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.103496 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.103479 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3dfe9b-104d-4a61-9766-205643d58666-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.109798 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.109779 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4vqn\" (UniqueName: \"kubernetes.io/projected/4e3dfe9b-104d-4a61-9766-205643d58666-kube-api-access-c4vqn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-crq56\" (UID: \"4e3dfe9b-104d-4a61-9766-205643d58666\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.121498 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.121477 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:30:54.254424 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:54.254398 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56"] Jul 05 13:30:54.256259 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:30:54.256227 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e3dfe9b_104d_4a61_9766_205643d58666.slice/crio-6c44ebd5bb49495c2c5b5c74edab548cb481d333022a5e47f7b5d4348ff5227f WatchSource:0}: Error finding container 6c44ebd5bb49495c2c5b5c74edab548cb481d333022a5e47f7b5d4348ff5227f: Status 404 returned error can't find the container with id 6c44ebd5bb49495c2c5b5c74edab548cb481d333022a5e47f7b5d4348ff5227f Jul 05 13:30:55.155971 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:55.155931 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" event={"ID":"4e3dfe9b-104d-4a61-9766-205643d58666","Type":"ContainerStarted","Data":"b2e6fb2da7920a2d790e314734aa6d80de37f301c54750c8b7e4c61ff8d1d5c7"} Jul 05 13:30:55.156338 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:55.155991 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" event={"ID":"4e3dfe9b-104d-4a61-9766-205643d58666","Type":"ContainerStarted","Data":"6c44ebd5bb49495c2c5b5c74edab548cb481d333022a5e47f7b5d4348ff5227f"} Jul 05 13:30:55.228896 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:30:55.228855 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" podStartSLOduration=2.073726764 podStartE2EDuration="2.228841489s" totalImagesPullingTime="155.114725ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:30:55.227271909 +0000 UTC m=+759.855637663" watchObservedRunningTime="2026-07-05 13:30:55.228841489 +0000 UTC m=+759.857207278" Jul 05 13:31:04.121903 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:31:04.121868 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:31:04.122452 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:31:04.121954 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:31:04.134354 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:31:04.134330 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:31:04.203709 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:31:04.203668 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-crq56" Jul 05 13:45:00.143830 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.143798 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29720985-hwbsq"] Jul 05 13:45:00.146951 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.146935 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" Jul 05 13:45:00.149588 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.149567 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-9bvwj\"" Jul 05 13:45:00.170614 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.170591 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720985-hwbsq"] Jul 05 13:45:00.190962 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.190936 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqw7l\" (UniqueName: \"kubernetes.io/projected/f51cf3a2-96c0-42c9-a809-6133e4df2c88-kube-api-access-dqw7l\") pod \"maas-api-key-cleanup-29720985-hwbsq\" (UID: \"f51cf3a2-96c0-42c9-a809-6133e4df2c88\") " pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" Jul 05 13:45:00.292097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.292076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqw7l\" (UniqueName: \"kubernetes.io/projected/f51cf3a2-96c0-42c9-a809-6133e4df2c88-kube-api-access-dqw7l\") pod \"maas-api-key-cleanup-29720985-hwbsq\" (UID: \"f51cf3a2-96c0-42c9-a809-6133e4df2c88\") " pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" Jul 05 13:45:00.301623 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.301608 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqw7l\" (UniqueName: \"kubernetes.io/projected/f51cf3a2-96c0-42c9-a809-6133e4df2c88-kube-api-access-dqw7l\") pod \"maas-api-key-cleanup-29720985-hwbsq\" (UID: \"f51cf3a2-96c0-42c9-a809-6133e4df2c88\") " pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" Jul 05 13:45:00.461548 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.461477 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" Jul 05 13:45:00.590861 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.590840 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29720985-hwbsq"] Jul 05 13:45:00.592794 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:45:00.592763 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf51cf3a2_96c0_42c9_a809_6133e4df2c88.slice/crio-d85f4e5d92bc79c882b401da0ead0101e4004ec9fa289e1809cdba0cd7af964b WatchSource:0}: Error finding container d85f4e5d92bc79c882b401da0ead0101e4004ec9fa289e1809cdba0cd7af964b: Status 404 returned error can't find the container with id d85f4e5d92bc79c882b401da0ead0101e4004ec9fa289e1809cdba0cd7af964b Jul 05 13:45:00.594578 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.594558 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 13:45:00.917350 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:00.917316 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" event={"ID":"f51cf3a2-96c0-42c9-a809-6133e4df2c88","Type":"ContainerStarted","Data":"d85f4e5d92bc79c882b401da0ead0101e4004ec9fa289e1809cdba0cd7af964b"} Jul 05 13:45:03.932408 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:03.932370 2579 generic.go:356] "Generic (PLEG): container finished" podID="f51cf3a2-96c0-42c9-a809-6133e4df2c88" containerID="eae183ca59100ac26088c70db90516b1b4251ba2c5c5622cb78657d8808d494e" exitCode=0 Jul 05 13:45:03.932859 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:03.932454 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" event={"ID":"f51cf3a2-96c0-42c9-a809-6133e4df2c88","Type":"ContainerDied","Data":"eae183ca59100ac26088c70db90516b1b4251ba2c5c5622cb78657d8808d494e"} Jul 05 13:45:05.069115 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:05.069091 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" Jul 05 13:45:05.134612 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:05.134577 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqw7l\" (UniqueName: \"kubernetes.io/projected/f51cf3a2-96c0-42c9-a809-6133e4df2c88-kube-api-access-dqw7l\") pod \"f51cf3a2-96c0-42c9-a809-6133e4df2c88\" (UID: \"f51cf3a2-96c0-42c9-a809-6133e4df2c88\") " Jul 05 13:45:05.136728 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:05.136705 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f51cf3a2-96c0-42c9-a809-6133e4df2c88-kube-api-access-dqw7l" (OuterVolumeSpecName: "kube-api-access-dqw7l") pod "f51cf3a2-96c0-42c9-a809-6133e4df2c88" (UID: "f51cf3a2-96c0-42c9-a809-6133e4df2c88"). InnerVolumeSpecName "kube-api-access-dqw7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:45:05.235463 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:05.235401 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqw7l\" (UniqueName: \"kubernetes.io/projected/f51cf3a2-96c0-42c9-a809-6133e4df2c88-kube-api-access-dqw7l\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:45:05.942559 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:05.942530 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" event={"ID":"f51cf3a2-96c0-42c9-a809-6133e4df2c88","Type":"ContainerDied","Data":"d85f4e5d92bc79c882b401da0ead0101e4004ec9fa289e1809cdba0cd7af964b"} Jul 05 13:45:05.942748 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:05.942576 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d85f4e5d92bc79c882b401da0ead0101e4004ec9fa289e1809cdba0cd7af964b" Jul 05 13:45:05.942748 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:05.942544 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29720985-hwbsq" Jul 05 13:45:55.798215 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.796749 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b"] Jul 05 13:45:55.798215 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.797091 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f51cf3a2-96c0-42c9-a809-6133e4df2c88" containerName="cleanup" Jul 05 13:45:55.798215 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.797103 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f51cf3a2-96c0-42c9-a809-6133e4df2c88" containerName="cleanup" Jul 05 13:45:55.798215 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.797159 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f51cf3a2-96c0-42c9-a809-6133e4df2c88" containerName="cleanup" Jul 05 13:45:55.800118 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.800093 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.802646 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.802622 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-72122384-openshift-default-dockercfg-l7dl5\"" Jul 05 13:45:55.819631 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.819605 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b"] Jul 05 13:45:55.855254 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.855215 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istiod-ca-cert\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.855254 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.855260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-token\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.855403 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.855356 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-credential-socket\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.855403 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.855385 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-socket\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.855494 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.855407 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-data\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.855494 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.855423 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-kube-api-access-57gbg\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.855494 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.855455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-envoy\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.855494 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.855473 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-podinfo\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.855494 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.855491 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-certs\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.956532 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.956499 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-envoy\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.956716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.956539 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-podinfo\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.956716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.956569 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-certs\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.956716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.956620 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istiod-ca-cert\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.956716 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.956652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-token\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.956924 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.956781 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-credential-socket\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.956924 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.956812 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-socket\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.956924 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.956844 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-data\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.956924 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.956868 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-kube-api-access-57gbg\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.957148 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.957055 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-certs\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.957327 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.957305 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-socket\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.957413 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.957352 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-data\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.957413 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.957399 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istiod-ca-cert\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.957567 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.957544 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-credential-socket\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.959057 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.959035 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-envoy\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.959136 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.959126 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-podinfo\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.966610 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.966589 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-kube-api-access-57gbg\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:55.967185 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:55.967159 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-token\") pod \"e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:56.110789 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:56.110762 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:56.260729 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:56.260702 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b"] Jul 05 13:45:56.263573 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:45:56.263547 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b6b4ed_f0ab_47b3_be02_f89f62dc69c1.slice/crio-55bd06ffa712d581564f9aa1dbe6290c3fcda38f6162a2d2d468336670d4c061 WatchSource:0}: Error finding container 55bd06ffa712d581564f9aa1dbe6290c3fcda38f6162a2d2d468336670d4c061: Status 404 returned error can't find the container with id 55bd06ffa712d581564f9aa1dbe6290c3fcda38f6162a2d2d468336670d4c061 Jul 05 13:45:56.265511 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:56.265476 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:45:56.265584 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:56.265556 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:45:56.265621 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:56.265599 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:45:57.188408 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:57.188376 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" event={"ID":"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1","Type":"ContainerStarted","Data":"4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f"} Jul 05 13:45:57.188784 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:57.188429 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" event={"ID":"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1","Type":"ContainerStarted","Data":"55bd06ffa712d581564f9aa1dbe6290c3fcda38f6162a2d2d468336670d4c061"} Jul 05 13:45:58.111396 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:58.111358 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:58.116103 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:58.116082 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:58.146220 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:58.146166 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" podStartSLOduration=3.146150443 podStartE2EDuration="3.146150443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:45:57.238555375 +0000 UTC m=+1661.866921129" watchObservedRunningTime="2026-07-05 13:45:58.146150443 +0000 UTC m=+1662.774516198" Jul 05 13:45:58.195355 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:58.195323 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:45:58.196054 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:45:58.196039 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:46:02.280671 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:02.280634 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b"] Jul 05 13:46:02.281134 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:02.280856 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" podUID="a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" containerName="istio-proxy" containerID="cri-o://4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f" gracePeriod=30 Jul 05 13:46:07.533412 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.533388 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:46:07.663837 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.663809 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-socket\") pod \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " Jul 05 13:46:07.664035 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.663893 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-podinfo\") pod \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " Jul 05 13:46:07.664035 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.663917 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-data\") pod \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " Jul 05 13:46:07.664035 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.663940 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-credential-socket\") pod \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " Jul 05 13:46:07.664035 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.663965 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-envoy\") pod \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " Jul 05 13:46:07.664035 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664003 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istiod-ca-cert\") pod \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " Jul 05 13:46:07.664292 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664066 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-token\") pod \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " Jul 05 13:46:07.664292 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664099 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-certs\") pod \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " Jul 05 13:46:07.664292 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664115 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" (UID: "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:07.664292 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664133 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-kube-api-access-57gbg\") pod \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\" (UID: \"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1\") " Jul 05 13:46:07.664292 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664249 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-data" (OuterVolumeSpecName: "istio-data") pod "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" (UID: "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:07.664518 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664289 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" (UID: "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:07.664518 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664422 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" (UID: "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:07.664518 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664481 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" (UID: "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:46:07.664694 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664598 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:07.664694 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664615 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:07.664694 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664630 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:07.664694 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664643 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:07.664694 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.664657 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:07.666479 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.666449 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-kube-api-access-57gbg" (OuterVolumeSpecName: "kube-api-access-57gbg") pod "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" (UID: "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1"). InnerVolumeSpecName "kube-api-access-57gbg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:07.666650 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.666480 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" (UID: "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:46:07.666650 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.666547 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" (UID: "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:07.666650 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.666635 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-token" (OuterVolumeSpecName: "istio-token") pod "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" (UID: "a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:07.765781 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.765748 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:07.765781 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.765776 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:07.765984 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.765789 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-kube-api-access-57gbg\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:07.765984 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:07.765802 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:08.241298 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.241263 2579 generic.go:356] "Generic (PLEG): container finished" podID="a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" containerID="4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f" exitCode=0 Jul 05 13:46:08.241502 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.241310 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" Jul 05 13:46:08.241502 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.241314 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" event={"ID":"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1","Type":"ContainerDied","Data":"4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f"} Jul 05 13:46:08.241502 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.241382 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b" event={"ID":"a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1","Type":"ContainerDied","Data":"55bd06ffa712d581564f9aa1dbe6290c3fcda38f6162a2d2d468336670d4c061"} Jul 05 13:46:08.241502 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.241411 2579 scope.go:117] "RemoveContainer" containerID="4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f" Jul 05 13:46:08.250610 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.250591 2579 scope.go:117] "RemoveContainer" containerID="4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f" Jul 05 13:46:08.250874 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:46:08.250855 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f\": container with ID starting with 4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f not found: ID does not exist" containerID="4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f" Jul 05 13:46:08.250961 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.250881 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f"} err="failed to get container status \"4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f\": rpc error: code = NotFound desc = could not find container \"4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f\": container with ID starting with 4bf112613f172cbaa753f1e5c7c0c5015dd367fd4f4f39cf4816cba29e891c1f not found: ID does not exist" Jul 05 13:46:08.267043 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.267017 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b"] Jul 05 13:46:08.286218 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.286200 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-72122384-openshift-default-7d4bccd4d5-b4q2b"] Jul 05 13:46:08.811787 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.811757 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5"] Jul 05 13:46:08.812142 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.812099 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" containerName="istio-proxy" Jul 05 13:46:08.812142 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.812110 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" containerName="istio-proxy" Jul 05 13:46:08.812216 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.812167 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" containerName="istio-proxy" Jul 05 13:46:08.816581 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.816558 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:08.819408 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.819388 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-cfdb1b35-openshift-default-dockercfg-8dzhn\"" Jul 05 13:46:08.831804 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.831784 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5"] Jul 05 13:46:08.976663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.976628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-socket\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:08.976663 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.976665 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-certs\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:08.976938 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.976709 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-data\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:08.976938 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.976833 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9d00e7-937d-4642-b128-f8d083300c55-istio-podinfo\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:08.976938 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.976862 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9d00e7-937d-4642-b128-f8d083300c55-istiod-ca-cert\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:08.976938 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.976888 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-credential-socket\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:08.976938 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.976905 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-envoy\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:08.977149 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.976981 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-istio-token\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:08.977149 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:08.977002 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8f59\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-kube-api-access-f8f59\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.077984 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.077918 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-socket\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.077984 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.077949 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-certs\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.077984 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.077971 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-data\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078219 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.077992 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9d00e7-937d-4642-b128-f8d083300c55-istio-podinfo\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078219 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078010 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9d00e7-937d-4642-b128-f8d083300c55-istiod-ca-cert\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078219 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078034 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-credential-socket\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078219 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078056 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-envoy\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078219 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-istio-token\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078219 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078150 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8f59\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-kube-api-access-f8f59\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078513 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078346 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-data\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078513 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078371 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-socket\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078513 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078446 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-certs\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078704 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078513 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-credential-socket\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.078865 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.078842 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9d00e7-937d-4642-b128-f8d083300c55-istiod-ca-cert\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.080463 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.080443 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-envoy\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.080809 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.080791 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9d00e7-937d-4642-b128-f8d083300c55-istio-podinfo\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.089616 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.089592 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-istio-token\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.090241 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.090222 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8f59\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-kube-api-access-f8f59\") pod \"e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.127193 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.127169 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:09.263717 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.263691 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5"] Jul 05 13:46:09.265577 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:46:09.265555 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9d00e7_937d_4642_b128_f8d083300c55.slice/crio-505be067ce70f1aadb20d42a52a6f8635ebf655a59b7ef543debf9bb35776d6d WatchSource:0}: Error finding container 505be067ce70f1aadb20d42a52a6f8635ebf655a59b7ef543debf9bb35776d6d: Status 404 returned error can't find the container with id 505be067ce70f1aadb20d42a52a6f8635ebf655a59b7ef543debf9bb35776d6d Jul 05 13:46:09.267397 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.267366 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:46:09.267485 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.267427 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:46:09.267485 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.267452 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:46:09.893150 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:09.893119 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1" path="/var/lib/kubelet/pods/a0b6b4ed-f0ab-47b3-be02-f89f62dc69c1/volumes" Jul 05 13:46:10.252746 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:10.252703 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" event={"ID":"fc9d00e7-937d-4642-b128-f8d083300c55","Type":"ContainerStarted","Data":"4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9"} Jul 05 13:46:10.252923 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:10.252758 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" event={"ID":"fc9d00e7-937d-4642-b128-f8d083300c55","Type":"ContainerStarted","Data":"505be067ce70f1aadb20d42a52a6f8635ebf655a59b7ef543debf9bb35776d6d"} Jul 05 13:46:10.303476 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:10.303428 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" podStartSLOduration=2.3034135620000002 podStartE2EDuration="2.303413562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:46:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:46:10.302790545 +0000 UTC m=+1674.931156301" watchObservedRunningTime="2026-07-05 13:46:10.303413562 +0000 UTC m=+1674.931779318" Jul 05 13:46:11.128072 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:11.128041 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:11.132849 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:11.132814 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:11.257278 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:11.257242 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:11.258062 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:11.258044 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:21.489284 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:21.489249 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5"] Jul 05 13:46:21.489781 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:21.489455 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" podUID="fc9d00e7-937d-4642-b128-f8d083300c55" containerName="istio-proxy" containerID="cri-o://4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9" gracePeriod=30 Jul 05 13:46:26.259507 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.259474 2579 patch_prober.go:28] interesting pod/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 05 13:46:26.260008 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.259566 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" podUID="fc9d00e7-937d-4642-b128-f8d083300c55" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 13:46:26.738038 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.738014 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:26.836329 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836296 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8f59\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-kube-api-access-f8f59\") pod \"fc9d00e7-937d-4642-b128-f8d083300c55\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " Jul 05 13:46:26.836492 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836358 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-credential-socket\") pod \"fc9d00e7-937d-4642-b128-f8d083300c55\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " Jul 05 13:46:26.836492 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836417 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-istio-token\") pod \"fc9d00e7-937d-4642-b128-f8d083300c55\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " Jul 05 13:46:26.836492 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836438 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-data\") pod \"fc9d00e7-937d-4642-b128-f8d083300c55\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " Jul 05 13:46:26.836492 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836480 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-envoy\") pod \"fc9d00e7-937d-4642-b128-f8d083300c55\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " Jul 05 13:46:26.836699 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836507 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9d00e7-937d-4642-b128-f8d083300c55-istiod-ca-cert\") pod \"fc9d00e7-937d-4642-b128-f8d083300c55\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " Jul 05 13:46:26.836699 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836536 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9d00e7-937d-4642-b128-f8d083300c55-istio-podinfo\") pod \"fc9d00e7-937d-4642-b128-f8d083300c55\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " Jul 05 13:46:26.836699 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836570 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-socket\") pod \"fc9d00e7-937d-4642-b128-f8d083300c55\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " Jul 05 13:46:26.836699 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836591 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-certs\") pod \"fc9d00e7-937d-4642-b128-f8d083300c55\" (UID: \"fc9d00e7-937d-4642-b128-f8d083300c55\") " Jul 05 13:46:26.836931 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836754 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc9d00e7-937d-4642-b128-f8d083300c55" (UID: "fc9d00e7-937d-4642-b128-f8d083300c55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:26.836931 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836776 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc9d00e7-937d-4642-b128-f8d083300c55" (UID: "fc9d00e7-937d-4642-b128-f8d083300c55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:26.837051 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836971 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:26.837051 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.836991 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:26.837051 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.837004 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc9d00e7-937d-4642-b128-f8d083300c55" (UID: "fc9d00e7-937d-4642-b128-f8d083300c55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:26.837051 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.837025 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc9d00e7-937d-4642-b128-f8d083300c55" (UID: "fc9d00e7-937d-4642-b128-f8d083300c55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:26.837267 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.837185 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc9d00e7-937d-4642-b128-f8d083300c55-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc9d00e7-937d-4642-b128-f8d083300c55" (UID: "fc9d00e7-937d-4642-b128-f8d083300c55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:46:26.838749 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.838726 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-kube-api-access-f8f59" (OuterVolumeSpecName: "kube-api-access-f8f59") pod "fc9d00e7-937d-4642-b128-f8d083300c55" (UID: "fc9d00e7-937d-4642-b128-f8d083300c55"). InnerVolumeSpecName "kube-api-access-f8f59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:26.838942 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.838919 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc9d00e7-937d-4642-b128-f8d083300c55-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc9d00e7-937d-4642-b128-f8d083300c55" (UID: "fc9d00e7-937d-4642-b128-f8d083300c55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:46:26.839177 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.839155 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc9d00e7-937d-4642-b128-f8d083300c55" (UID: "fc9d00e7-937d-4642-b128-f8d083300c55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:26.839274 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.839163 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc9d00e7-937d-4642-b128-f8d083300c55" (UID: "fc9d00e7-937d-4642-b128-f8d083300c55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:26.938012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.937947 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:26.938012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.937968 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:26.938012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.937977 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9d00e7-937d-4642-b128-f8d083300c55-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:26.938012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.937984 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9d00e7-937d-4642-b128-f8d083300c55-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:26.938012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.937991 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:26.938012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.937999 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9d00e7-937d-4642-b128-f8d083300c55-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:26.938012 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:26.938007 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8f59\" (UniqueName: \"kubernetes.io/projected/fc9d00e7-937d-4642-b128-f8d083300c55-kube-api-access-f8f59\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:27.329882 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.329845 2579 generic.go:356] "Generic (PLEG): container finished" podID="fc9d00e7-937d-4642-b128-f8d083300c55" containerID="4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9" exitCode=0 Jul 05 13:46:27.330280 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.329895 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" Jul 05 13:46:27.330280 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.329922 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" event={"ID":"fc9d00e7-937d-4642-b128-f8d083300c55","Type":"ContainerDied","Data":"4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9"} Jul 05 13:46:27.330280 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.329982 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5" event={"ID":"fc9d00e7-937d-4642-b128-f8d083300c55","Type":"ContainerDied","Data":"505be067ce70f1aadb20d42a52a6f8635ebf655a59b7ef543debf9bb35776d6d"} Jul 05 13:46:27.330280 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.330013 2579 scope.go:117] "RemoveContainer" containerID="4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9" Jul 05 13:46:27.345924 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.345905 2579 scope.go:117] "RemoveContainer" containerID="4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9" Jul 05 13:46:27.346182 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:46:27.346161 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9\": container with ID starting with 4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9 not found: ID does not exist" containerID="4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9" Jul 05 13:46:27.346238 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.346190 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9"} err="failed to get container status \"4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9\": rpc error: code = NotFound desc = could not find container \"4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9\": container with ID starting with 4f2cf361e9647a962ab7d06e3d1642f055c0bcf989b8579860e4adcb49393cc9 not found: ID does not exist" Jul 05 13:46:27.355530 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.355509 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5"] Jul 05 13:46:27.358993 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.358973 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-cfdb1b35-openshift-default-f6ccb5c6c-nqlb5"] Jul 05 13:46:27.679015 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.678941 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll"] Jul 05 13:46:27.679539 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.679516 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9d00e7-937d-4642-b128-f8d083300c55" containerName="istio-proxy" Jul 05 13:46:27.679743 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.679722 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9d00e7-937d-4642-b128-f8d083300c55" containerName="istio-proxy" Jul 05 13:46:27.679878 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.679864 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc9d00e7-937d-4642-b128-f8d083300c55" containerName="istio-proxy" Jul 05 13:46:27.684394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.684375 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.688077 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.688055 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-5d6488dd-openshift-default-dockercfg-5mfcq\"" Jul 05 13:46:27.698399 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.698379 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll"] Jul 05 13:46:27.745701 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.745656 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-credential-socket\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.745834 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.745783 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-data\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.745834 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.745811 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-socket\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.745834 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.745831 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-certs\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.745974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.745849 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-envoy\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.745974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.745883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqtmn\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-kube-api-access-cqtmn\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.745974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.745912 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-token\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.745974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.745953 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-podinfo\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.745974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.745971 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istiod-ca-cert\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847388 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847348 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-credential-socket\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847576 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-data\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847576 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847481 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-socket\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847576 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847505 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-certs\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847576 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-envoy\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847576 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847557 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqtmn\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-kube-api-access-cqtmn\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847587 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-token\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-podinfo\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847651 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istiod-ca-cert\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847825 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-credential-socket\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.847894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847863 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-socket\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.848184 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.847908 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-certs\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.848184 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.848079 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-data\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.848547 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.848525 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istiod-ca-cert\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.850041 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.850011 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-envoy\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.850231 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.850213 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-podinfo\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.859007 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.858983 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqtmn\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-kube-api-access-cqtmn\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.859247 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.859223 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-token\") pod \"e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:27.893828 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.893802 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc9d00e7-937d-4642-b128-f8d083300c55" path="/var/lib/kubelet/pods/fc9d00e7-937d-4642-b128-f8d083300c55/volumes" Jul 05 13:46:27.994538 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:27.994465 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:28.132133 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:28.132110 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll"] Jul 05 13:46:28.133975 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:46:28.133948 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4345f10c_ebcc_4b9f_bfab_9a82fdb88e00.slice/crio-d84dd795499c26f1d8e97f879300dd212f4a055e080bfc3f1ba2776627c6c3b0 WatchSource:0}: Error finding container d84dd795499c26f1d8e97f879300dd212f4a055e080bfc3f1ba2776627c6c3b0: Status 404 returned error can't find the container with id d84dd795499c26f1d8e97f879300dd212f4a055e080bfc3f1ba2776627c6c3b0 Jul 05 13:46:28.136071 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:28.136040 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:46:28.136157 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:28.136124 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:46:28.136157 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:28.136151 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:46:28.337430 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:28.337383 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" event={"ID":"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00","Type":"ContainerStarted","Data":"e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9"} Jul 05 13:46:28.337928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:28.337444 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" event={"ID":"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00","Type":"ContainerStarted","Data":"d84dd795499c26f1d8e97f879300dd212f4a055e080bfc3f1ba2776627c6c3b0"} Jul 05 13:46:28.362812 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:28.362757 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" podStartSLOduration=1.362736585 podStartE2EDuration="1.362736585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:46:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:46:28.360638231 +0000 UTC m=+1692.989003987" watchObservedRunningTime="2026-07-05 13:46:28.362736585 +0000 UTC m=+1692.991102341" Jul 05 13:46:28.994884 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:28.994850 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:30.000164 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:30.000133 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:30.347256 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:30.347224 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:30.348214 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:30.348197 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:33.618286 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:33.618251 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll"] Jul 05 13:46:33.618705 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:33.618482 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" podUID="4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" containerName="istio-proxy" containerID="cri-o://e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9" gracePeriod=30 Jul 05 13:46:38.867104 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.867079 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:38.948081 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.947998 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-envoy\") pod \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " Jul 05 13:46:38.948081 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948031 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-data\") pod \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " Jul 05 13:46:38.948081 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948050 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-podinfo\") pod \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " Jul 05 13:46:38.948347 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948087 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-certs\") pod \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " Jul 05 13:46:38.948347 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948114 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-token\") pod \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " Jul 05 13:46:38.948347 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948139 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqtmn\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-kube-api-access-cqtmn\") pod \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " Jul 05 13:46:38.948347 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948158 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-socket\") pod \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " Jul 05 13:46:38.948347 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948247 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-credential-socket\") pod \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " Jul 05 13:46:38.948347 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948267 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istiod-ca-cert\") pod \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\" (UID: \"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00\") " Jul 05 13:46:38.948649 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948442 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-data" (OuterVolumeSpecName: "istio-data") pod "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" (UID: "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:38.948649 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948467 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" (UID: "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:38.948794 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948703 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:38.948794 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948724 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:38.948794 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948733 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" (UID: "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:38.948794 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948710 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" (UID: "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:38.949018 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.948996 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" (UID: "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:46:38.950468 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.950442 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" (UID: "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:46:38.950772 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.950742 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-kube-api-access-cqtmn" (OuterVolumeSpecName: "kube-api-access-cqtmn") pod "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" (UID: "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00"). InnerVolumeSpecName "kube-api-access-cqtmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:38.950919 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.950902 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" (UID: "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:46:38.950991 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:38.950974 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-token" (OuterVolumeSpecName: "istio-token") pod "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" (UID: "4345f10c-ebcc-4b9f-bfab-9a82fdb88e00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:46:39.049161 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.049132 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:39.049161 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.049157 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqtmn\" (UniqueName: \"kubernetes.io/projected/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-kube-api-access-cqtmn\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:39.049161 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.049167 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:39.049375 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.049175 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:39.049375 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.049184 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:39.049375 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.049192 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:39.049375 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.049199 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:46:39.386388 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.386353 2579 generic.go:356] "Generic (PLEG): container finished" podID="4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" containerID="e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9" exitCode=0 Jul 05 13:46:39.386563 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.386394 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" Jul 05 13:46:39.386563 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.386431 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" event={"ID":"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00","Type":"ContainerDied","Data":"e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9"} Jul 05 13:46:39.386563 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.386485 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll" event={"ID":"4345f10c-ebcc-4b9f-bfab-9a82fdb88e00","Type":"ContainerDied","Data":"d84dd795499c26f1d8e97f879300dd212f4a055e080bfc3f1ba2776627c6c3b0"} Jul 05 13:46:39.386563 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.386508 2579 scope.go:117] "RemoveContainer" containerID="e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9" Jul 05 13:46:39.395698 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.395659 2579 scope.go:117] "RemoveContainer" containerID="e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9" Jul 05 13:46:39.395978 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:46:39.395958 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9\": container with ID starting with e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9 not found: ID does not exist" containerID="e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9" Jul 05 13:46:39.396037 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.395989 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9"} err="failed to get container status \"e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9\": rpc error: code = NotFound desc = could not find container \"e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9\": container with ID starting with e16fa7ef89e6eb7488c6e7f08ee66ae3afbde5643e3da724a0340b814d275dc9 not found: ID does not exist" Jul 05 13:46:39.409573 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.409546 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll"] Jul 05 13:46:39.413930 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.413908 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-5d6488dd-openshift-default-646bb4dd6c-sp6ll"] Jul 05 13:46:39.893791 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:46:39.893760 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" path="/var/lib/kubelet/pods/4345f10c-ebcc-4b9f-bfab-9a82fdb88e00/volumes" Jul 05 13:48:18.916439 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:18.916408 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd"] Jul 05 13:48:18.916873 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:18.916753 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" containerName="istio-proxy" Jul 05 13:48:18.916873 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:18.916764 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" containerName="istio-proxy" Jul 05 13:48:18.916873 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:18.916827 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4345f10c-ebcc-4b9f-bfab-9a82fdb88e00" containerName="istio-proxy" Jul 05 13:48:18.919619 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:18.919603 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:18.925658 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:18.925638 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fcfb85dc-openshift-default-dockercfg-4frv6\"" Jul 05 13:48:18.977839 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:18.977799 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd"] Jul 05 13:48:19.040311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.040282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-credential-socket\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.040535 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.040336 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-token\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.040535 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.040401 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-data\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.040535 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.040438 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-socket\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.040535 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.040490 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-podinfo\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.040535 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.040506 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmt62\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-kube-api-access-cmt62\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.040782 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.040540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5b829c-a49c-4c53-a696-5d6cab412f07-istiod-ca-cert\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.040782 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.040563 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-envoy\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.040782 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.040586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-certs\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141368 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141341 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-credential-socket\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141546 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-token\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141546 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-data\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141546 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141433 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-socket\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141546 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141467 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-podinfo\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141546 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141492 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmt62\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-kube-api-access-cmt62\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141546 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5b829c-a49c-4c53-a696-5d6cab412f07-istiod-ca-cert\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141920 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-envoy\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141920 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141586 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-certs\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141920 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141804 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-credential-socket\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141920 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141891 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-data\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.141920 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.141918 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-socket\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.142170 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.142081 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-certs\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.142216 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.142188 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5b829c-a49c-4c53-a696-5d6cab412f07-istiod-ca-cert\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.144010 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.143985 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-envoy\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.144202 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.144185 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-podinfo\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.150122 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.150100 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-token\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.159194 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.159168 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmt62\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-kube-api-access-cmt62\") pod \"e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.230186 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.230128 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:19.378795 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.378767 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd"] Jul 05 13:48:19.381247 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:48:19.381214 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5b829c_a49c_4c53_a696_5d6cab412f07.slice/crio-93213b4adb817a865df91ceae26969359e9ff0b8962ca8d5f8c751f3d71cdb00 WatchSource:0}: Error finding container 93213b4adb817a865df91ceae26969359e9ff0b8962ca8d5f8c751f3d71cdb00: Status 404 returned error can't find the container with id 93213b4adb817a865df91ceae26969359e9ff0b8962ca8d5f8c751f3d71cdb00 Jul 05 13:48:19.383327 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.383290 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:48:19.383430 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.383371 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:48:19.383430 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.383416 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:48:19.850298 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.850262 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" event={"ID":"ab5b829c-a49c-4c53-a696-5d6cab412f07","Type":"ContainerStarted","Data":"7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750"} Jul 05 13:48:19.850477 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:19.850326 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" event={"ID":"ab5b829c-a49c-4c53-a696-5d6cab412f07","Type":"ContainerStarted","Data":"93213b4adb817a865df91ceae26969359e9ff0b8962ca8d5f8c751f3d71cdb00"} Jul 05 13:48:20.230311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:20.230231 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:20.235149 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:20.235128 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:20.264380 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:20.264328 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" podStartSLOduration=2.264307918 podStartE2EDuration="2.264307918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:48:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:48:19.889167089 +0000 UTC m=+1804.517532844" watchObservedRunningTime="2026-07-05 13:48:20.264307918 +0000 UTC m=+1804.892673673" Jul 05 13:48:20.855073 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:20.855036 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:20.856075 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:20.856057 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:34.924897 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:34.924856 2579 generic.go:356] "Generic (PLEG): container finished" podID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerID="7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750" exitCode=0 Jul 05 13:48:34.925302 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:34.924933 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" event={"ID":"ab5b829c-a49c-4c53-a696-5d6cab412f07","Type":"ContainerDied","Data":"7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750"} Jul 05 13:48:34.925378 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:34.925359 2579 scope.go:117] "RemoveContainer" containerID="7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750" Jul 05 13:48:34.927004 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:34.926963 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:48:34.927250 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:34.927211 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:48:34.927390 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:34.927267 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:48:35.855087 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:35.855048 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:35.931178 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:35.931143 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" event={"ID":"ab5b829c-a49c-4c53-a696-5d6cab412f07","Type":"ContainerStarted","Data":"cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098"} Jul 05 13:48:36.230628 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:36.230539 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:36.231860 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:36.231834 2579 patch_prober.go:28] interesting pod/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 05 13:48:36.231992 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:36.231881 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 05 13:48:37.230776 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:37.230741 2579 patch_prober.go:28] interesting pod/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 05 13:48:37.231147 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:37.230792 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 05 13:48:38.231532 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:38.231497 2579 patch_prober.go:28] interesting pod/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 05 13:48:38.231939 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:38.231549 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 05 13:48:39.230613 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:39.230577 2579 patch_prober.go:28] interesting pod/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 05 13:48:39.230812 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:39.230629 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 05 13:48:40.235589 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:40.235561 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:40.236027 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:40.235937 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:40.236631 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:40.236612 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:41.644742 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:41.644712 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd"] Jul 05 13:48:41.964351 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:41.964273 2579 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" secret="" err="secret \"e2e-mt-fcfb85dc-openshift-default-dockercfg-4frv6\" not found" Jul 05 13:48:42.968324 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:42.968280 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" containerID="cri-o://cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098" gracePeriod=30 Jul 05 13:48:48.219031 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.219004 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:48.316310 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316286 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-envoy\") pod \"ab5b829c-a49c-4c53-a696-5d6cab412f07\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " Jul 05 13:48:48.316451 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316333 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-token\") pod \"ab5b829c-a49c-4c53-a696-5d6cab412f07\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " Jul 05 13:48:48.316451 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316351 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-podinfo\") pod \"ab5b829c-a49c-4c53-a696-5d6cab412f07\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " Jul 05 13:48:48.316451 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316378 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5b829c-a49c-4c53-a696-5d6cab412f07-istiod-ca-cert\") pod \"ab5b829c-a49c-4c53-a696-5d6cab412f07\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " Jul 05 13:48:48.316451 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316396 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-certs\") pod \"ab5b829c-a49c-4c53-a696-5d6cab412f07\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " Jul 05 13:48:48.316451 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316426 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-data\") pod \"ab5b829c-a49c-4c53-a696-5d6cab412f07\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " Jul 05 13:48:48.316744 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316479 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmt62\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-kube-api-access-cmt62\") pod \"ab5b829c-a49c-4c53-a696-5d6cab412f07\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " Jul 05 13:48:48.316744 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316507 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-credential-socket\") pod \"ab5b829c-a49c-4c53-a696-5d6cab412f07\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " Jul 05 13:48:48.316744 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316545 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-socket\") pod \"ab5b829c-a49c-4c53-a696-5d6cab412f07\" (UID: \"ab5b829c-a49c-4c53-a696-5d6cab412f07\") " Jul 05 13:48:48.316897 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316785 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab5b829c-a49c-4c53-a696-5d6cab412f07" (UID: "ab5b829c-a49c-4c53-a696-5d6cab412f07"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:48.316897 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.316870 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab5b829c-a49c-4c53-a696-5d6cab412f07-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab5b829c-a49c-4c53-a696-5d6cab412f07" (UID: "ab5b829c-a49c-4c53-a696-5d6cab412f07"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:48:48.317079 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.317058 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab5b829c-a49c-4c53-a696-5d6cab412f07-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:48:48.317168 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.317084 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:48:48.317168 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.317121 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab5b829c-a49c-4c53-a696-5d6cab412f07" (UID: "ab5b829c-a49c-4c53-a696-5d6cab412f07"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:48.317168 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.317152 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab5b829c-a49c-4c53-a696-5d6cab412f07" (UID: "ab5b829c-a49c-4c53-a696-5d6cab412f07"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:48.317553 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.317525 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab5b829c-a49c-4c53-a696-5d6cab412f07" (UID: "ab5b829c-a49c-4c53-a696-5d6cab412f07"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:48.318825 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.318797 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab5b829c-a49c-4c53-a696-5d6cab412f07" (UID: "ab5b829c-a49c-4c53-a696-5d6cab412f07"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:48:48.319029 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.319006 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-kube-api-access-cmt62" (OuterVolumeSpecName: "kube-api-access-cmt62") pod "ab5b829c-a49c-4c53-a696-5d6cab412f07" (UID: "ab5b829c-a49c-4c53-a696-5d6cab412f07"). InnerVolumeSpecName "kube-api-access-cmt62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:48:48.319201 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.319173 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab5b829c-a49c-4c53-a696-5d6cab412f07" (UID: "ab5b829c-a49c-4c53-a696-5d6cab412f07"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:48:48.319301 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.319183 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab5b829c-a49c-4c53-a696-5d6cab412f07" (UID: "ab5b829c-a49c-4c53-a696-5d6cab412f07"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:48:48.418202 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.418173 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:48:48.418202 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.418197 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:48:48.418202 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.418206 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:48:48.418405 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.418215 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:48:48.418405 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.418223 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmt62\" (UniqueName: \"kubernetes.io/projected/ab5b829c-a49c-4c53-a696-5d6cab412f07-kube-api-access-cmt62\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:48:48.418405 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.418232 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:48:48.418405 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.418240 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab5b829c-a49c-4c53-a696-5d6cab412f07-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:48:48.996259 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.996225 2579 generic.go:356] "Generic (PLEG): container finished" podID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerID="cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098" exitCode=0 Jul 05 13:48:48.996487 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.996263 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" event={"ID":"ab5b829c-a49c-4c53-a696-5d6cab412f07","Type":"ContainerDied","Data":"cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098"} Jul 05 13:48:48.996487 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.996271 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" Jul 05 13:48:48.996487 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.996300 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd" event={"ID":"ab5b829c-a49c-4c53-a696-5d6cab412f07","Type":"ContainerDied","Data":"93213b4adb817a865df91ceae26969359e9ff0b8962ca8d5f8c751f3d71cdb00"} Jul 05 13:48:48.996487 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:48.996321 2579 scope.go:117] "RemoveContainer" containerID="cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098" Jul 05 13:48:49.005854 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:49.005838 2579 scope.go:117] "RemoveContainer" containerID="7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750" Jul 05 13:48:49.013519 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:49.013504 2579 scope.go:117] "RemoveContainer" containerID="cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098" Jul 05 13:48:49.013785 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:48:49.013765 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098\": container with ID starting with cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098 not found: ID does not exist" containerID="cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098" Jul 05 13:48:49.013856 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:49.013792 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098"} err="failed to get container status \"cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098\": rpc error: code = NotFound desc = could not find container \"cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098\": container with ID starting with cf187ff99a951be76f227cbea302cd687d07e6ff724abb6b8da050da9178e098 not found: ID does not exist" Jul 05 13:48:49.013856 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:49.013807 2579 scope.go:117] "RemoveContainer" containerID="7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750" Jul 05 13:48:49.014034 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:48:49.014017 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750\": container with ID starting with 7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750 not found: ID does not exist" containerID="7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750" Jul 05 13:48:49.014081 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:49.014040 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750"} err="failed to get container status \"7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750\": rpc error: code = NotFound desc = could not find container \"7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750\": container with ID starting with 7aabc24e95b074c722aef5e738bb5d3ed49019ca64d401d77f17a64dad337750 not found: ID does not exist" Jul 05 13:48:49.021058 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:49.021039 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd"] Jul 05 13:48:49.027047 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:49.027026 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fcfb85dc-openshift-default-5fb98cc487-r2lcd"] Jul 05 13:48:49.893713 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:48:49.893659 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" path="/var/lib/kubelet/pods/ab5b829c-a49c-4c53-a696-5d6cab412f07/volumes" Jul 05 13:49:02.582914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.582877 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml"] Jul 05 13:49:02.583393 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.583219 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" Jul 05 13:49:02.583393 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.583231 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" Jul 05 13:49:02.583393 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.583245 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" Jul 05 13:49:02.583393 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.583251 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" Jul 05 13:49:02.583393 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.583323 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" Jul 05 13:49:02.583393 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.583335 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab5b829c-a49c-4c53-a696-5d6cab412f07" containerName="istio-proxy" Jul 05 13:49:02.588009 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.587987 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.590768 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.590744 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6a0254a3-openshift-default-dockercfg-gjcd8\"" Jul 05 13:49:02.600461 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.600439 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml"] Jul 05 13:49:02.743444 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.743411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-credential-socket\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.743636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.743460 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-envoy\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.743636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.743490 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-socket\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.743636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.743523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istiod-ca-cert\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.743636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.743540 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-data\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.743636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.743589 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-certs\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.743636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.743624 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-podinfo\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.743933 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.743655 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slb6s\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-kube-api-access-slb6s\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.743933 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.743747 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-token\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845140 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-token\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845206 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-credential-socket\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845403 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845234 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-envoy\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845403 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845258 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-socket\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845403 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istiod-ca-cert\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845561 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845422 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-data\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845561 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845469 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-certs\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845561 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845498 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-podinfo\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845561 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slb6s\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-kube-api-access-slb6s\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845796 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845707 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-socket\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845796 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845735 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-credential-socket\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.845902 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.845810 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-certs\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.846058 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.846041 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istiod-ca-cert\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.846357 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.846333 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-data\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.847743 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.847723 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-envoy\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.848048 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.848029 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-podinfo\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.857649 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.857629 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-token\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.860694 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.860652 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slb6s\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-kube-api-access-slb6s\") pod \"e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:02.899792 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:02.899764 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:03.045773 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:03.045744 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml"] Jul 05 13:49:03.048371 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:49:03.048346 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b8c2be_63cb_46f7_8dc8_e83ff2ca3699.slice/crio-9f5714e96a5cd0cbd59db447e57204c8c9697115a37cd6f6a41360b7e543b0d8 WatchSource:0}: Error finding container 9f5714e96a5cd0cbd59db447e57204c8c9697115a37cd6f6a41360b7e543b0d8: Status 404 returned error can't find the container with id 9f5714e96a5cd0cbd59db447e57204c8c9697115a37cd6f6a41360b7e543b0d8 Jul 05 13:49:03.050563 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:03.050526 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:49:03.050659 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:03.050603 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:49:03.050659 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:03.050655 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:49:03.061512 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:03.061491 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" event={"ID":"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699","Type":"ContainerStarted","Data":"9f5714e96a5cd0cbd59db447e57204c8c9697115a37cd6f6a41360b7e543b0d8"} Jul 05 13:49:04.067748 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:04.067713 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" event={"ID":"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699","Type":"ContainerStarted","Data":"8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f"} Jul 05 13:49:04.097459 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:04.097414 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" podStartSLOduration=2.097399742 podStartE2EDuration="2.097399742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:49:04.094709633 +0000 UTC m=+1848.723075382" watchObservedRunningTime="2026-07-05 13:49:04.097399742 +0000 UTC m=+1848.725765499" Jul 05 13:49:04.900167 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:04.900127 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:04.905376 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:04.905351 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:05.072129 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:05.072098 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:05.072956 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:05.072923 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:20.073823 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:20.073789 2579 patch_prober.go:28] interesting pod/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 05 13:49:20.074224 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:20.073841 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 05 13:49:20.149234 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:20.149162 2579 generic.go:356] "Generic (PLEG): container finished" podID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerID="8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f" exitCode=0 Jul 05 13:49:20.149365 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:20.149236 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" event={"ID":"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699","Type":"ContainerDied","Data":"8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f"} Jul 05 13:49:20.149627 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:20.149613 2579 scope.go:117] "RemoveContainer" containerID="8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f" Jul 05 13:49:20.150595 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:20.150572 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:49:20.150645 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:20.150636 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:49:20.150696 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:20.150669 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:49:21.156881 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:21.156849 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" event={"ID":"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699","Type":"ContainerStarted","Data":"5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae"} Jul 05 13:49:21.900334 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:21.900292 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:21.900602 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:21.900575 2579 patch_prober.go:28] interesting pod/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 05 13:49:21.900709 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:21.900635 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 05 13:49:22.900251 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:22.900210 2579 patch_prober.go:28] interesting pod/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 05 13:49:22.900733 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:22.900273 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 05 13:49:23.900884 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:23.900850 2579 patch_prober.go:28] interesting pod/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 05 13:49:23.901357 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:23.900922 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 05 13:49:24.900722 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:24.900660 2579 patch_prober.go:28] interesting pod/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 05 13:49:24.901023 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:24.900992 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 05 13:49:25.904075 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:25.904044 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:25.904450 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:25.904300 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:49:25.906437 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:49:25.906405 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:50:38.520101 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:38.520069 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml"] Jul 05 13:50:38.520612 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:38.520277 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" containerID="cri-o://5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae" gracePeriod=30 Jul 05 13:50:40.906699 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:40.906644 2579 patch_prober.go:28] interesting pod/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 05 13:50:40.907067 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:40.906720 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 13:50:43.771394 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.771371 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:50:43.938338 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938311 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-data\") pod \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " Jul 05 13:50:43.938338 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938343 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-certs\") pod \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " Jul 05 13:50:43.938621 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938367 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-envoy\") pod \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " Jul 05 13:50:43.938621 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938389 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-token\") pod \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " Jul 05 13:50:43.938621 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938439 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slb6s\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-kube-api-access-slb6s\") pod \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " Jul 05 13:50:43.938621 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938496 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-socket\") pod \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " Jul 05 13:50:43.938621 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938532 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istiod-ca-cert\") pod \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " Jul 05 13:50:43.938621 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938554 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-podinfo\") pod \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " Jul 05 13:50:43.938621 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938578 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-credential-socket\") pod \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\" (UID: \"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699\") " Jul 05 13:50:43.939045 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938657 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" (UID: "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:50:43.939045 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.938956 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:50:43.939662 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.939637 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" (UID: "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:50:43.940119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.940078 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" (UID: "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:50:43.940237 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.940178 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" (UID: "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:50:43.940420 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.940394 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" (UID: "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:50:43.941289 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.941263 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" (UID: "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:50:43.941289 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.941273 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" (UID: "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:50:43.941518 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.941502 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" (UID: "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:50:43.943289 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:43.943262 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-kube-api-access-slb6s" (OuterVolumeSpecName: "kube-api-access-slb6s") pod "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" (UID: "f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699"). InnerVolumeSpecName "kube-api-access-slb6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:50:44.039888 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.039850 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:50:44.039888 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.039885 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:50:44.039888 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.039899 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:50:44.040097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.039913 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slb6s\" (UniqueName: \"kubernetes.io/projected/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-kube-api-access-slb6s\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:50:44.040097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.039925 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:50:44.040097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.039937 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:50:44.040097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.039949 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:50:44.040097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.039960 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:50:44.533795 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.533762 2579 generic.go:356] "Generic (PLEG): container finished" podID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerID="5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae" exitCode=0 Jul 05 13:50:44.533973 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.533823 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" Jul 05 13:50:44.533973 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.533858 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" event={"ID":"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699","Type":"ContainerDied","Data":"5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae"} Jul 05 13:50:44.533973 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.533916 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml" event={"ID":"f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699","Type":"ContainerDied","Data":"9f5714e96a5cd0cbd59db447e57204c8c9697115a37cd6f6a41360b7e543b0d8"} Jul 05 13:50:44.533973 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.533955 2579 scope.go:117] "RemoveContainer" containerID="5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae" Jul 05 13:50:44.542948 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.542928 2579 scope.go:117] "RemoveContainer" containerID="8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f" Jul 05 13:50:44.550571 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.550554 2579 scope.go:117] "RemoveContainer" containerID="5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae" Jul 05 13:50:44.550859 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:50:44.550839 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae\": container with ID starting with 5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae not found: ID does not exist" containerID="5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae" Jul 05 13:50:44.550927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.550869 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae"} err="failed to get container status \"5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae\": rpc error: code = NotFound desc = could not find container \"5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae\": container with ID starting with 5e1ae8a1d41d9a6ae6d2a4d8ed86c2fa354637ebca7d1ad4df65d87df9d334ae not found: ID does not exist" Jul 05 13:50:44.550927 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.550889 2579 scope.go:117] "RemoveContainer" containerID="8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f" Jul 05 13:50:44.551122 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:50:44.551104 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f\": container with ID starting with 8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f not found: ID does not exist" containerID="8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f" Jul 05 13:50:44.551185 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.551132 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f"} err="failed to get container status \"8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f\": rpc error: code = NotFound desc = could not find container \"8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f\": container with ID starting with 8af79664367c509ee02eb462ec0ab8859dcd4e9a470eeee9939a9e7a3cecd42f not found: ID does not exist" Jul 05 13:50:44.558598 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.558560 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml"] Jul 05 13:50:44.560467 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:44.560434 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6a0254a3-openshift-default-77699b65b7-mdgml"] Jul 05 13:50:45.898925 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:50:45.898893 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" path="/var/lib/kubelet/pods/f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699/volumes" Jul 05 13:51:35.138582 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.138550 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw"] Jul 05 13:51:35.139060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.138965 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" Jul 05 13:51:35.139060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.138982 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" Jul 05 13:51:35.139060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.138992 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" Jul 05 13:51:35.139060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.138998 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" Jul 05 13:51:35.139203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.139067 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" Jul 05 13:51:35.139203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.139077 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b8c2be-63cb-46f7-8dc8-e83ff2ca3699" containerName="istio-proxy" Jul 05 13:51:35.142052 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.142034 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.145873 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.145853 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3c48d7-openshift-default-dockercfg-d2mld\"" Jul 05 13:51:35.176431 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.176402 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-certs\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.176611 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.176450 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-token\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.176611 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.176488 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-data\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.176788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.176602 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-credential-socket\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.176788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.176651 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-socket\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.176788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.176722 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-podinfo\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.176788 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.176766 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b84fb0-b917-400a-848e-a71edd9263d6-istiod-ca-cert\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.177267 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.177181 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-envoy\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.177513 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.177493 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p9fm\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-kube-api-access-5p9fm\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.179633 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.179611 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw"] Jul 05 13:51:35.278700 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.278639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-podinfo\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.278871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.278711 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b84fb0-b917-400a-848e-a71edd9263d6-istiod-ca-cert\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.278871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.278742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-envoy\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.278871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.278784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p9fm\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-kube-api-access-5p9fm\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.278871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.278831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-certs\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.278871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.278857 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-token\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.279140 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.278912 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-data\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.279140 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.278959 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-credential-socket\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.279140 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.278998 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-socket\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.279304 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.279271 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-certs\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.279379 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.279356 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-data\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.279438 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.279404 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-credential-socket\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.279500 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.279480 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-socket\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.279569 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.279546 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b84fb0-b917-400a-848e-a71edd9263d6-istiod-ca-cert\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.281751 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.281724 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-envoy\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.281971 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.281954 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-podinfo\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.291473 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.291448 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-token\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.291918 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.291896 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p9fm\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-kube-api-access-5p9fm\") pod \"e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.452348 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.452267 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:35.601523 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.601482 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw"] Jul 05 13:51:35.603689 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:51:35.603646 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b84fb0_b917_400a_848e_a71edd9263d6.slice/crio-69f42c03f8ee04603a1d800cdf5dbe7789308517173d604745023e618bd9b93a WatchSource:0}: Error finding container 69f42c03f8ee04603a1d800cdf5dbe7789308517173d604745023e618bd9b93a: Status 404 returned error can't find the container with id 69f42c03f8ee04603a1d800cdf5dbe7789308517173d604745023e618bd9b93a Jul 05 13:51:35.605828 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.605796 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:35.605917 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.605861 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:35.605917 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.605896 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:35.765871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.765785 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" event={"ID":"a3b84fb0-b917-400a-848e-a71edd9263d6","Type":"ContainerStarted","Data":"3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1"} Jul 05 13:51:35.765871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.765843 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" event={"ID":"a3b84fb0-b917-400a-848e-a71edd9263d6","Type":"ContainerStarted","Data":"69f42c03f8ee04603a1d800cdf5dbe7789308517173d604745023e618bd9b93a"} Jul 05 13:51:35.791098 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:35.791042 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" podStartSLOduration=0.791021716 podStartE2EDuration="791.021716ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:51:35.787590866 +0000 UTC m=+2000.415956629" watchObservedRunningTime="2026-07-05 13:51:35.791021716 +0000 UTC m=+2000.419387472" Jul 05 13:51:36.452439 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:36.452406 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:36.457328 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:36.457308 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:36.770912 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:36.770819 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:36.771801 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:36.771780 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:43.010010 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.009980 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2"] Jul 05 13:51:43.013522 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.013505 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.016212 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.016192 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-114a16-openshift-default-dockercfg-4l9rx\"" Jul 05 13:51:43.028195 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.028172 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2"] Jul 05 13:51:43.051349 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.051321 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-socket\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.051454 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.051353 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-data\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.051454 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.051427 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-token\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.051454 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.051450 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-certs\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.051584 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.051476 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-credential-socket\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.051584 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.051507 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-envoy\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.051584 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.051529 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istiod-ca-cert\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.051584 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.051557 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45brd\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-kube-api-access-45brd\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.051741 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.051585 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-podinfo\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.152849 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.152813 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-envoy\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153042 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.152858 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istiod-ca-cert\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153042 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.152890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45brd\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-kube-api-access-45brd\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153042 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.152929 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-podinfo\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153042 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.152976 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-socket\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153042 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.153006 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-data\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153362 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.153087 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-token\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153362 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.153124 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-certs\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153362 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.153152 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-credential-socket\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153570 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.153383 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-socket\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153570 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.153445 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-data\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153785 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.153758 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istiod-ca-cert\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153850 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.153823 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-certs\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.153960 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.153942 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-credential-socket\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.155481 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.155458 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-envoy\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.155571 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.155513 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-podinfo\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.162950 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.162930 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-token\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.163402 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.163379 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45brd\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-kube-api-access-45brd\") pod \"e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.324291 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.324267 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:43.496765 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.496740 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2"] Jul 05 13:51:43.498762 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:51:43.498731 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf34a990_5ce0_4fed_86c2_fd33d24e16a6.slice/crio-91e317bd9d9658111482e9698284f7022288725915f5eaf940e4f3b5e8cfa3e0 WatchSource:0}: Error finding container 91e317bd9d9658111482e9698284f7022288725915f5eaf940e4f3b5e8cfa3e0: Status 404 returned error can't find the container with id 91e317bd9d9658111482e9698284f7022288725915f5eaf940e4f3b5e8cfa3e0 Jul 05 13:51:43.501065 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.501034 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:43.501172 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.501102 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:43.501172 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.501138 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:43.805718 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.805664 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" event={"ID":"df34a990-5ce0-4fed-86c2-fd33d24e16a6","Type":"ContainerStarted","Data":"fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9"} Jul 05 13:51:43.805865 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.805733 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" event={"ID":"df34a990-5ce0-4fed-86c2-fd33d24e16a6","Type":"ContainerStarted","Data":"91e317bd9d9658111482e9698284f7022288725915f5eaf940e4f3b5e8cfa3e0"} Jul 05 13:51:43.835247 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:43.835194 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" podStartSLOduration=1.835178604 podStartE2EDuration="1.835178604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:51:43.832375718 +0000 UTC m=+2008.460741472" watchObservedRunningTime="2026-07-05 13:51:43.835178604 +0000 UTC m=+2008.463544360" Jul 05 13:51:44.324998 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:44.324974 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:44.329654 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:44.329630 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:44.810848 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:44.810820 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:44.811907 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:44.811869 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:51:52.812979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:52.812926 2579 patch_prober.go:28] interesting pod/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 05 13:51:52.813649 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:52.813617 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 05 13:51:53.858405 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:53.858374 2579 generic.go:356] "Generic (PLEG): container finished" podID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerID="3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1" exitCode=0 Jul 05 13:51:53.858848 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:53.858453 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" event={"ID":"a3b84fb0-b917-400a-848e-a71edd9263d6","Type":"ContainerDied","Data":"3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1"} Jul 05 13:51:53.858890 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:53.858872 2579 scope.go:117] "RemoveContainer" containerID="3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1" Jul 05 13:51:53.859951 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:53.859928 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:53.860030 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:53.859991 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:53.860068 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:53.860032 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:54.864868 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:54.864837 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" event={"ID":"a3b84fb0-b917-400a-848e-a71edd9263d6","Type":"ContainerStarted","Data":"43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836"} Jul 05 13:51:55.453602 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:55.453571 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:55.453799 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:55.453605 2579 patch_prober.go:28] interesting pod/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 05 13:51:55.453799 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:55.453649 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 05 13:51:56.453471 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:56.453431 2579 patch_prober.go:28] interesting pod/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 05 13:51:56.453914 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:56.453491 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 05 13:51:57.452930 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:57.452889 2579 patch_prober.go:28] interesting pod/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 05 13:51:57.453100 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:57.452959 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 05 13:51:58.453085 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:58.453044 2579 patch_prober.go:28] interesting pod/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 05 13:51:58.453487 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:58.453114 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 05 13:51:59.456343 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.456316 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:59.457881 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.456593 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:59.457881 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.457159 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:51:59.812493 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.812457 2579 patch_prober.go:28] interesting pod/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 05 13:51:59.812654 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.812517 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 05 13:51:59.890188 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.890154 2579 generic.go:356] "Generic (PLEG): container finished" podID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerID="fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9" exitCode=0 Jul 05 13:51:59.893278 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.893253 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" event={"ID":"df34a990-5ce0-4fed-86c2-fd33d24e16a6","Type":"ContainerDied","Data":"fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9"} Jul 05 13:51:59.893606 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.893583 2579 scope.go:117] "RemoveContainer" containerID="fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9" Jul 05 13:51:59.894562 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.894533 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:59.894652 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.894617 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:51:59.894740 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:51:59.894667 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:52:00.896749 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:00.896712 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" event={"ID":"df34a990-5ce0-4fed-86c2-fd33d24e16a6","Type":"ContainerStarted","Data":"d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da"} Jul 05 13:52:01.325184 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:01.325141 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:52:01.325429 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:01.325406 2579 patch_prober.go:28] interesting pod/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 05 13:52:01.325488 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:01.325469 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 05 13:52:02.325637 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:02.325597 2579 patch_prober.go:28] interesting pod/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 05 13:52:02.326117 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:02.325661 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 05 13:52:03.324843 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:03.324802 2579 patch_prober.go:28] interesting pod/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 05 13:52:03.325021 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:03.324871 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 05 13:52:04.325499 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:04.325461 2579 patch_prober.go:28] interesting pod/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 05 13:52:04.325922 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:04.325522 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 05 13:52:05.325618 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:05.325582 2579 patch_prober.go:28] interesting pod/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 05 13:52:05.326048 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:05.325649 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 05 13:52:06.329399 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:06.329367 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:52:06.329812 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:06.329717 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:52:06.330380 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:52:06.330362 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:53:43.358558 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:43.358523 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw"] Jul 05 13:53:43.359054 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:43.358793 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" containerID="cri-o://43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836" gracePeriod=30 Jul 05 13:53:44.459406 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:44.459368 2579 patch_prober.go:28] interesting pod/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 05 13:53:44.459916 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:44.459448 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 13:53:48.612102 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.612076 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:53:48.775837 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.775750 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-credential-socket\") pod \"a3b84fb0-b917-400a-848e-a71edd9263d6\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " Jul 05 13:53:48.775837 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.775788 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b84fb0-b917-400a-848e-a71edd9263d6-istiod-ca-cert\") pod \"a3b84fb0-b917-400a-848e-a71edd9263d6\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " Jul 05 13:53:48.775837 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.775817 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-podinfo\") pod \"a3b84fb0-b917-400a-848e-a71edd9263d6\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " Jul 05 13:53:48.776108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.775855 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-envoy\") pod \"a3b84fb0-b917-400a-848e-a71edd9263d6\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " Jul 05 13:53:48.776108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.775884 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p9fm\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-kube-api-access-5p9fm\") pod \"a3b84fb0-b917-400a-848e-a71edd9263d6\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " Jul 05 13:53:48.776108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.775901 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-certs\") pod \"a3b84fb0-b917-400a-848e-a71edd9263d6\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " Jul 05 13:53:48.776108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.775916 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-socket\") pod \"a3b84fb0-b917-400a-848e-a71edd9263d6\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " Jul 05 13:53:48.776108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.775982 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-token\") pod \"a3b84fb0-b917-400a-848e-a71edd9263d6\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " Jul 05 13:53:48.776108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.776046 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-data\") pod \"a3b84fb0-b917-400a-848e-a71edd9263d6\" (UID: \"a3b84fb0-b917-400a-848e-a71edd9263d6\") " Jul 05 13:53:48.776416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.776105 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3b84fb0-b917-400a-848e-a71edd9263d6" (UID: "a3b84fb0-b917-400a-848e-a71edd9263d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:48.776416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.776243 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3b84fb0-b917-400a-848e-a71edd9263d6" (UID: "a3b84fb0-b917-400a-848e-a71edd9263d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:48.776416 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.776257 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b84fb0-b917-400a-848e-a71edd9263d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3b84fb0-b917-400a-848e-a71edd9263d6" (UID: "a3b84fb0-b917-400a-848e-a71edd9263d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:53:48.776572 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.776468 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:48.776572 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.776483 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b84fb0-b917-400a-848e-a71edd9263d6-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:48.776572 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.776473 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3b84fb0-b917-400a-848e-a71edd9263d6" (UID: "a3b84fb0-b917-400a-848e-a71edd9263d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:48.776572 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.776493 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:48.777405 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.777376 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3b84fb0-b917-400a-848e-a71edd9263d6" (UID: "a3b84fb0-b917-400a-848e-a71edd9263d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:48.778393 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.778368 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3b84fb0-b917-400a-848e-a71edd9263d6" (UID: "a3b84fb0-b917-400a-848e-a71edd9263d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:53:48.778487 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.778386 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3b84fb0-b917-400a-848e-a71edd9263d6" (UID: "a3b84fb0-b917-400a-848e-a71edd9263d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:53:48.778487 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.778391 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-kube-api-access-5p9fm" (OuterVolumeSpecName: "kube-api-access-5p9fm") pod "a3b84fb0-b917-400a-848e-a71edd9263d6" (UID: "a3b84fb0-b917-400a-848e-a71edd9263d6"). InnerVolumeSpecName "kube-api-access-5p9fm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:53:48.778615 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.778599 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3b84fb0-b917-400a-848e-a71edd9263d6" (UID: "a3b84fb0-b917-400a-848e-a71edd9263d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:48.877312 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.877278 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p9fm\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-kube-api-access-5p9fm\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:48.877312 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.877306 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:48.877312 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.877315 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:48.877312 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.877322 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:48.877563 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.877332 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:48.877563 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:48.877339 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b84fb0-b917-400a-848e-a71edd9263d6-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:49.412303 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.412264 2579 generic.go:356] "Generic (PLEG): container finished" podID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerID="43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836" exitCode=0 Jul 05 13:53:49.412507 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.412317 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" Jul 05 13:53:49.412507 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.412346 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" event={"ID":"a3b84fb0-b917-400a-848e-a71edd9263d6","Type":"ContainerDied","Data":"43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836"} Jul 05 13:53:49.412507 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.412407 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw" event={"ID":"a3b84fb0-b917-400a-848e-a71edd9263d6","Type":"ContainerDied","Data":"69f42c03f8ee04603a1d800cdf5dbe7789308517173d604745023e618bd9b93a"} Jul 05 13:53:49.412507 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.412438 2579 scope.go:117] "RemoveContainer" containerID="43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836" Jul 05 13:53:49.422584 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.422566 2579 scope.go:117] "RemoveContainer" containerID="3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1" Jul 05 13:53:49.430958 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.430942 2579 scope.go:117] "RemoveContainer" containerID="43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836" Jul 05 13:53:49.431187 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:53:49.431171 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836\": container with ID starting with 43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836 not found: ID does not exist" containerID="43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836" Jul 05 13:53:49.431232 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.431195 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836"} err="failed to get container status \"43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836\": rpc error: code = NotFound desc = could not find container \"43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836\": container with ID starting with 43cf9b6e92e38ac819d1e7ed9bf69db4965ecb5a0b0b15e8055c644c466c3836 not found: ID does not exist" Jul 05 13:53:49.431232 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.431211 2579 scope.go:117] "RemoveContainer" containerID="3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1" Jul 05 13:53:49.431406 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:53:49.431393 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1\": container with ID starting with 3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1 not found: ID does not exist" containerID="3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1" Jul 05 13:53:49.431450 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.431410 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1"} err="failed to get container status \"3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1\": rpc error: code = NotFound desc = could not find container \"3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1\": container with ID starting with 3e053124528f9711353f99b4501b4a463621c0d53adc1c0f0b2c18857e504ca1 not found: ID does not exist" Jul 05 13:53:49.435765 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.435741 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw"] Jul 05 13:53:49.439490 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.439470 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3c48d7-openshift-default-674f7447d7-7jksw"] Jul 05 13:53:49.893535 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:49.893500 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" path="/var/lib/kubelet/pods/a3b84fb0-b917-400a-848e-a71edd9263d6/volumes" Jul 05 13:53:50.635184 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:50.635154 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2"] Jul 05 13:53:50.635400 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:50.635359 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" containerID="cri-o://d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da" gracePeriod=30 Jul 05 13:53:51.331863 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.331829 2579 patch_prober.go:28] interesting pod/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 05 13:53:51.332256 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.331917 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 13:53:51.537065 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.537028 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq"] Jul 05 13:53:51.537459 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.537441 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" Jul 05 13:53:51.537502 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.537465 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" Jul 05 13:53:51.537591 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.537579 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" Jul 05 13:53:51.537626 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.537599 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" Jul 05 13:53:51.542144 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.542122 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.545448 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.545428 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e72226-openshift-default-dockercfg-5p7xv\"" Jul 05 13:53:51.554630 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.554605 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq"] Jul 05 13:53:51.702030 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.701936 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-socket\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.702030 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.701997 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-token\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.702030 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.702020 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-envoy\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.702288 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.702052 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-credential-socket\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.702288 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.702074 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvbkv\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-kube-api-access-wvbkv\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.702288 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.702144 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-podinfo\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.702288 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.702178 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istiod-ca-cert\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.702288 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.702215 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-certs\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.702288 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.702239 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-data\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.802759 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.802727 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-socket\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.802943 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.802774 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-token\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.802943 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.802798 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-envoy\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.802943 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.802831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-credential-socket\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.802943 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.802854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvbkv\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-kube-api-access-wvbkv\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.802943 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.802907 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-podinfo\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.803203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.802961 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istiod-ca-cert\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.803203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.802997 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-certs\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.803203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.803035 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-data\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.803203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.803119 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-socket\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.803203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.803146 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-credential-socket\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.803470 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.803395 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-certs\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.803518 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.803504 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-data\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.803868 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.803841 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istiod-ca-cert\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.805639 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.805606 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-envoy\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.805830 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.805813 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-podinfo\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.818930 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.818900 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvbkv\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-kube-api-access-wvbkv\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.831035 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.831004 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-token\") pod \"e2e-api-a-e72226-openshift-default-5c54665c9-678qq\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.852507 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.852466 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:51.996253 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:51.996219 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq"] Jul 05 13:53:51.998750 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:53:51.998714 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8879bfa5_2eb2_454e_92fe_7f1ce1b62912.slice/crio-caa49d281892b604a5c602d3d15583bea6d530e116149c5dd6ad08e3334c337a WatchSource:0}: Error finding container caa49d281892b604a5c602d3d15583bea6d530e116149c5dd6ad08e3334c337a: Status 404 returned error can't find the container with id caa49d281892b604a5c602d3d15583bea6d530e116149c5dd6ad08e3334c337a Jul 05 13:53:52.000841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:52.000809 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:52.000946 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:52.000880 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:52.000946 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:52.000908 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:53:52.429187 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:52.429150 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" event={"ID":"8879bfa5-2eb2-454e-92fe-7f1ce1b62912","Type":"ContainerStarted","Data":"f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c"} Jul 05 13:53:52.429593 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:52.429206 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" event={"ID":"8879bfa5-2eb2-454e-92fe-7f1ce1b62912","Type":"ContainerStarted","Data":"caa49d281892b604a5c602d3d15583bea6d530e116149c5dd6ad08e3334c337a"} Jul 05 13:53:52.452517 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:52.452459 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" podStartSLOduration=1.45244255 podStartE2EDuration="1.45244255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:53:52.4508018 +0000 UTC m=+2137.079167556" watchObservedRunningTime="2026-07-05 13:53:52.45244255 +0000 UTC m=+2137.080808304" Jul 05 13:53:52.852871 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:52.852839 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:53.858075 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:53.858043 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:54.438301 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:54.438273 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:54.439311 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:54.439289 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:53:55.889782 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.889757 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:53:55.942066 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.941984 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-socket\") pod \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " Jul 05 13:53:55.942066 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942021 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istiod-ca-cert\") pod \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " Jul 05 13:53:55.942066 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942049 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45brd\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-kube-api-access-45brd\") pod \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " Jul 05 13:53:55.942359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942079 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-podinfo\") pod \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " Jul 05 13:53:55.942359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942096 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-envoy\") pod \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " Jul 05 13:53:55.942359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942161 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-certs\") pod \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " Jul 05 13:53:55.942359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942186 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-credential-socket\") pod \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " Jul 05 13:53:55.942359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942210 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-token\") pod \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " Jul 05 13:53:55.942359 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942277 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df34a990-5ce0-4fed-86c2-fd33d24e16a6" (UID: "df34a990-5ce0-4fed-86c2-fd33d24e16a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:55.942720 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942438 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df34a990-5ce0-4fed-86c2-fd33d24e16a6" (UID: "df34a990-5ce0-4fed-86c2-fd33d24e16a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:53:55.942720 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942441 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df34a990-5ce0-4fed-86c2-fd33d24e16a6" (UID: "df34a990-5ce0-4fed-86c2-fd33d24e16a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:55.942720 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942549 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:55.942720 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942568 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:55.942720 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942582 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:55.942998 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.942730 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df34a990-5ce0-4fed-86c2-fd33d24e16a6" (UID: "df34a990-5ce0-4fed-86c2-fd33d24e16a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:55.944504 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.944469 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df34a990-5ce0-4fed-86c2-fd33d24e16a6" (UID: "df34a990-5ce0-4fed-86c2-fd33d24e16a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:55.944626 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.944531 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "df34a990-5ce0-4fed-86c2-fd33d24e16a6" (UID: "df34a990-5ce0-4fed-86c2-fd33d24e16a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:53:55.945041 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.945019 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-kube-api-access-45brd" (OuterVolumeSpecName: "kube-api-access-45brd") pod "df34a990-5ce0-4fed-86c2-fd33d24e16a6" (UID: "df34a990-5ce0-4fed-86c2-fd33d24e16a6"). InnerVolumeSpecName "kube-api-access-45brd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:53:55.945041 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:55.945027 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df34a990-5ce0-4fed-86c2-fd33d24e16a6" (UID: "df34a990-5ce0-4fed-86c2-fd33d24e16a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:53:56.042979 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.042947 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-data\") pod \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\" (UID: \"df34a990-5ce0-4fed-86c2-fd33d24e16a6\") " Jul 05 13:53:56.043199 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.043187 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45brd\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-kube-api-access-45brd\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:56.043244 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.043201 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:56.043244 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.043212 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:56.043244 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.043219 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:56.043244 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.043227 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:56.043581 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.043558 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "df34a990-5ce0-4fed-86c2-fd33d24e16a6" (UID: "df34a990-5ce0-4fed-86c2-fd33d24e16a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:53:56.143832 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.143804 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df34a990-5ce0-4fed-86c2-fd33d24e16a6-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:53:56.448922 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.448892 2579 generic.go:356] "Generic (PLEG): container finished" podID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerID="d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da" exitCode=0 Jul 05 13:53:56.449097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.448945 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" Jul 05 13:53:56.449097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.448969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" event={"ID":"df34a990-5ce0-4fed-86c2-fd33d24e16a6","Type":"ContainerDied","Data":"d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da"} Jul 05 13:53:56.449097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.449026 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2" event={"ID":"df34a990-5ce0-4fed-86c2-fd33d24e16a6","Type":"ContainerDied","Data":"91e317bd9d9658111482e9698284f7022288725915f5eaf940e4f3b5e8cfa3e0"} Jul 05 13:53:56.449097 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.449051 2579 scope.go:117] "RemoveContainer" containerID="d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da" Jul 05 13:53:56.458196 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.458181 2579 scope.go:117] "RemoveContainer" containerID="fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9" Jul 05 13:53:56.466262 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.466246 2579 scope.go:117] "RemoveContainer" containerID="d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da" Jul 05 13:53:56.466483 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:53:56.466464 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da\": container with ID starting with d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da not found: ID does not exist" containerID="d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da" Jul 05 13:53:56.466551 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.466495 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da"} err="failed to get container status \"d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da\": rpc error: code = NotFound desc = could not find container \"d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da\": container with ID starting with d563850d44ecc80a3d2fabc023a4b32b051224b866560ed3dd80915ad63620da not found: ID does not exist" Jul 05 13:53:56.466551 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.466520 2579 scope.go:117] "RemoveContainer" containerID="fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9" Jul 05 13:53:56.466774 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:53:56.466757 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9\": container with ID starting with fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9 not found: ID does not exist" containerID="fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9" Jul 05 13:53:56.466826 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.466779 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9"} err="failed to get container status \"fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9\": rpc error: code = NotFound desc = could not find container \"fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9\": container with ID starting with fc42c5c1cd72d216ced47dfdab28d54a355ea01c3dd0f640bd69404de7a320e9 not found: ID does not exist" Jul 05 13:53:56.472637 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.472615 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2"] Jul 05 13:53:56.474288 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:56.474270 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-114a16-openshift-default-785bbb9d6c-k8xk2"] Jul 05 13:53:57.894370 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:57.894334 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" path="/var/lib/kubelet/pods/df34a990-5ce0-4fed-86c2-fd33d24e16a6/volumes" Jul 05 13:53:59.739019 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.738983 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4"] Jul 05 13:53:59.739557 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.739475 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" Jul 05 13:53:59.739557 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.739492 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" Jul 05 13:53:59.739557 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.739505 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" Jul 05 13:53:59.739557 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.739511 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b84fb0-b917-400a-848e-a71edd9263d6" containerName="istio-proxy" Jul 05 13:53:59.739753 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.739605 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" Jul 05 13:53:59.743882 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.743863 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.746472 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.746452 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d0bf0c-openshift-default-dockercfg-mpfzp\"" Jul 05 13:53:59.753528 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.753510 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4"] Jul 05 13:53:59.766667 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.766606 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-podinfo\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.766790 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.766701 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgwjx\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-kube-api-access-hgwjx\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.766790 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.766731 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-envoy\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.766920 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.766813 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-token\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.766920 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.766911 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istiod-ca-cert\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.767034 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.766964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-certs\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.767034 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.767016 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-credential-socket\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.767144 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.767110 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-socket\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.767200 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.767178 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-data\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.867845 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.867798 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-data\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.867845 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.867846 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-podinfo\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868079 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.867877 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgwjx\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-kube-api-access-hgwjx\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868079 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.867898 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-envoy\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868079 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.867912 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-token\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868079 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.868015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istiod-ca-cert\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868079 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.868055 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-certs\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868364 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.868095 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-credential-socket\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868364 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.868148 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-socket\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868478 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.868411 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-data\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868537 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.868467 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-certs\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868537 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.868519 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-credential-socket\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868614 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.868541 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-socket\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.868749 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.868730 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istiod-ca-cert\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.870919 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.870896 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-envoy\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.871169 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.871150 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-podinfo\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.875891 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.875866 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-token\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:53:59.876133 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:53:59.876117 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgwjx\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-kube-api-access-hgwjx\") pod \"e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:00.055022 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:00.054986 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:00.196399 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:00.196373 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4"] Jul 05 13:54:00.197992 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:54:00.197967 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72fd3dd0_4fb3_4456_9674_7e2d2fdfb9c9.slice/crio-9eb3813a3a501fb41e93fef0cc5c9cda0e98649eea6db241c49c36e3bda846a5 WatchSource:0}: Error finding container 9eb3813a3a501fb41e93fef0cc5c9cda0e98649eea6db241c49c36e3bda846a5: Status 404 returned error can't find the container with id 9eb3813a3a501fb41e93fef0cc5c9cda0e98649eea6db241c49c36e3bda846a5 Jul 05 13:54:00.200454 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:00.200425 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:00.200522 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:00.200486 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:00.200522 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:00.200513 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:00.470512 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:00.470420 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" event={"ID":"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9","Type":"ContainerStarted","Data":"07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0"} Jul 05 13:54:00.470512 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:00.470484 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" event={"ID":"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9","Type":"ContainerStarted","Data":"9eb3813a3a501fb41e93fef0cc5c9cda0e98649eea6db241c49c36e3bda846a5"} Jul 05 13:54:00.490070 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:00.490017 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" podStartSLOduration=1.489996198 podStartE2EDuration="1.489996198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:53:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:54:00.488918166 +0000 UTC m=+2145.117283920" watchObservedRunningTime="2026-07-05 13:54:00.489996198 +0000 UTC m=+2145.118361952" Jul 05 13:54:01.055772 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:01.055736 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:01.061403 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:01.061376 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:01.475038 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:01.474955 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:01.475928 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:01.475910 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:08.509787 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:08.509754 2579 generic.go:356] "Generic (PLEG): container finished" podID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerID="f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c" exitCode=0 Jul 05 13:54:08.510210 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:08.509797 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" event={"ID":"8879bfa5-2eb2-454e-92fe-7f1ce1b62912","Type":"ContainerDied","Data":"f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c"} Jul 05 13:54:08.510276 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:08.510259 2579 scope.go:117] "RemoveContainer" containerID="f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c" Jul 05 13:54:08.511320 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:08.511289 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:08.511414 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:08.511355 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:08.511414 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:08.511393 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:09.439200 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:09.439169 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:54:09.520222 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:09.520184 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" event={"ID":"8879bfa5-2eb2-454e-92fe-7f1ce1b62912","Type":"ContainerStarted","Data":"7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92"} Jul 05 13:54:09.852737 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:09.852668 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:54:09.853922 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:09.853898 2579 patch_prober.go:28] interesting pod/e2e-api-a-e72226-openshift-default-5c54665c9-678qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 05 13:54:09.854108 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:09.853949 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 05 13:54:10.853642 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:10.853603 2579 patch_prober.go:28] interesting pod/e2e-api-a-e72226-openshift-default-5c54665c9-678qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 05 13:54:10.854054 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:10.853665 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 05 13:54:11.853038 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:11.853001 2579 patch_prober.go:28] interesting pod/e2e-api-a-e72226-openshift-default-5c54665c9-678qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 05 13:54:11.853213 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:11.853054 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 05 13:54:12.853770 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:12.853731 2579 patch_prober.go:28] interesting pod/e2e-api-a-e72226-openshift-default-5c54665c9-678qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 05 13:54:12.854173 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:12.853781 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 05 13:54:13.857383 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:13.857347 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:54:13.857805 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:13.857710 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:54:13.858386 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:13.858367 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:54:17.516887 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:17.516828 2579 patch_prober.go:28] interesting pod/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 05 13:54:17.519287 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:17.516900 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 05 13:54:18.568958 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:18.568920 2579 generic.go:356] "Generic (PLEG): container finished" podID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerID="07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0" exitCode=0 Jul 05 13:54:18.569434 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:18.568974 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" event={"ID":"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9","Type":"ContainerDied","Data":"07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0"} Jul 05 13:54:18.569434 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:18.569414 2579 scope.go:117] "RemoveContainer" containerID="07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0" Jul 05 13:54:18.570432 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:18.570402 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:18.570532 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:18.570471 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:18.570532 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:18.570504 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:19.574738 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:19.574703 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" event={"ID":"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9","Type":"ContainerStarted","Data":"ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1"} Jul 05 13:54:20.055574 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:20.055550 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:20.055808 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:20.055786 2579 patch_prober.go:28] interesting pod/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 05 13:54:20.055859 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:20.055839 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 05 13:54:21.055910 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:21.055871 2579 patch_prober.go:28] interesting pod/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 05 13:54:21.056377 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:21.055947 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 05 13:54:21.535163 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:21.535077 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq"] Jul 05 13:54:21.535640 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:21.535607 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" containerID="cri-o://7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92" gracePeriod=30 Jul 05 13:54:22.055941 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:22.055911 2579 patch_prober.go:28] interesting pod/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 05 13:54:22.056301 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:22.055965 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 05 13:54:23.056438 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:23.056399 2579 patch_prober.go:28] interesting pod/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 05 13:54:23.056860 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:23.056453 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 05 13:54:24.058528 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:24.058499 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:24.058982 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:24.058764 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:24.059346 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:24.059329 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:26.792576 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.792547 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:54:26.949237 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949199 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-envoy\") pod \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " Jul 05 13:54:26.949433 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949276 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-data\") pod \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " Jul 05 13:54:26.949433 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949303 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-certs\") pod \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " Jul 05 13:54:26.949433 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949332 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-socket\") pod \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " Jul 05 13:54:26.949433 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949416 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-podinfo\") pod \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " Jul 05 13:54:26.949609 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949454 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istiod-ca-cert\") pod \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " Jul 05 13:54:26.949609 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949529 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvbkv\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-kube-api-access-wvbkv\") pod \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " Jul 05 13:54:26.949609 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949559 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-token\") pod \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " Jul 05 13:54:26.949609 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949580 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-credential-socket\") pod \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\" (UID: \"8879bfa5-2eb2-454e-92fe-7f1ce1b62912\") " Jul 05 13:54:26.949868 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949628 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8879bfa5-2eb2-454e-92fe-7f1ce1b62912" (UID: "8879bfa5-2eb2-454e-92fe-7f1ce1b62912"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:26.949868 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949741 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8879bfa5-2eb2-454e-92fe-7f1ce1b62912" (UID: "8879bfa5-2eb2-454e-92fe-7f1ce1b62912"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:26.950009 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949885 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8879bfa5-2eb2-454e-92fe-7f1ce1b62912" (UID: "8879bfa5-2eb2-454e-92fe-7f1ce1b62912"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:26.950009 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949959 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:26.950009 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949973 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:26.950009 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.949994 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:26.950236 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.950215 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8879bfa5-2eb2-454e-92fe-7f1ce1b62912" (UID: "8879bfa5-2eb2-454e-92fe-7f1ce1b62912"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:54:26.950368 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.950301 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-data" (OuterVolumeSpecName: "istio-data") pod "8879bfa5-2eb2-454e-92fe-7f1ce1b62912" (UID: "8879bfa5-2eb2-454e-92fe-7f1ce1b62912"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:26.951875 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.951837 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-token" (OuterVolumeSpecName: "istio-token") pod "8879bfa5-2eb2-454e-92fe-7f1ce1b62912" (UID: "8879bfa5-2eb2-454e-92fe-7f1ce1b62912"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:54:26.951875 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.951864 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-kube-api-access-wvbkv" (OuterVolumeSpecName: "kube-api-access-wvbkv") pod "8879bfa5-2eb2-454e-92fe-7f1ce1b62912" (UID: "8879bfa5-2eb2-454e-92fe-7f1ce1b62912"). InnerVolumeSpecName "kube-api-access-wvbkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:54:26.952191 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.952158 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8879bfa5-2eb2-454e-92fe-7f1ce1b62912" (UID: "8879bfa5-2eb2-454e-92fe-7f1ce1b62912"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:26.952191 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:26.952166 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8879bfa5-2eb2-454e-92fe-7f1ce1b62912" (UID: "8879bfa5-2eb2-454e-92fe-7f1ce1b62912"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:54:27.050572 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.050536 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:27.050572 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.050570 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:27.050881 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.050584 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvbkv\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-kube-api-access-wvbkv\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:27.050881 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.050599 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:27.050881 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.050612 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:27.050881 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.050624 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8879bfa5-2eb2-454e-92fe-7f1ce1b62912-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:27.613725 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.613670 2579 generic.go:356] "Generic (PLEG): container finished" podID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerID="7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92" exitCode=0 Jul 05 13:54:27.613725 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.613710 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" event={"ID":"8879bfa5-2eb2-454e-92fe-7f1ce1b62912","Type":"ContainerDied","Data":"7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92"} Jul 05 13:54:27.613998 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.613740 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" Jul 05 13:54:27.613998 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.613778 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq" event={"ID":"8879bfa5-2eb2-454e-92fe-7f1ce1b62912","Type":"ContainerDied","Data":"caa49d281892b604a5c602d3d15583bea6d530e116149c5dd6ad08e3334c337a"} Jul 05 13:54:27.613998 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.613810 2579 scope.go:117] "RemoveContainer" containerID="7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92" Jul 05 13:54:27.623433 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.623416 2579 scope.go:117] "RemoveContainer" containerID="f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c" Jul 05 13:54:27.632428 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.632265 2579 scope.go:117] "RemoveContainer" containerID="7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92" Jul 05 13:54:27.632589 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:54:27.632569 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92\": container with ID starting with 7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92 not found: ID does not exist" containerID="7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92" Jul 05 13:54:27.632649 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.632597 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92"} err="failed to get container status \"7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92\": rpc error: code = NotFound desc = could not find container \"7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92\": container with ID starting with 7b916b861132cd06367171ee0de0a3ea42bfcc79c94482eb6114229927a8bf92 not found: ID does not exist" Jul 05 13:54:27.632649 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.632621 2579 scope.go:117] "RemoveContainer" containerID="f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c" Jul 05 13:54:27.632892 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:54:27.632872 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c\": container with ID starting with f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c not found: ID does not exist" containerID="f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c" Jul 05 13:54:27.632941 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.632900 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c"} err="failed to get container status \"f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c\": rpc error: code = NotFound desc = could not find container \"f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c\": container with ID starting with f436c4cc2102a5d49a08d089b45e0ed3c89dffe99dbb4c3e478cc099ca18159c not found: ID does not exist" Jul 05 13:54:27.638797 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.638775 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq"] Jul 05 13:54:27.642710 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.642671 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e72226-openshift-default-5c54665c9-678qq"] Jul 05 13:54:27.893536 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:27.893456 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" path="/var/lib/kubelet/pods/8879bfa5-2eb2-454e-92fe-7f1ce1b62912/volumes" Jul 05 13:54:28.609902 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:28.609870 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4"] Jul 05 13:54:28.610115 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:28.610092 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" containerID="cri-o://ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1" gracePeriod=30 Jul 05 13:54:29.492502 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492473 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq"] Jul 05 13:54:29.492894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492822 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" Jul 05 13:54:29.492894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492833 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" Jul 05 13:54:29.492894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492841 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" Jul 05 13:54:29.492894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492845 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" Jul 05 13:54:29.492894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492859 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" Jul 05 13:54:29.492894 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492864 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" Jul 05 13:54:29.493087 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492936 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" Jul 05 13:54:29.493087 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492942 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8879bfa5-2eb2-454e-92fe-7f1ce1b62912" containerName="istio-proxy" Jul 05 13:54:29.493087 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.492949 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="df34a990-5ce0-4fed-86c2-fd33d24e16a6" containerName="istio-proxy" Jul 05 13:54:29.497807 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.497788 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.500203 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.500186 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c2f889-openshift-default-dockercfg-gzbbk\"" Jul 05 13:54:29.508323 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.508303 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq"] Jul 05 13:54:29.677119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.677072 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-socket\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.677119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.677125 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-certs\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.677381 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.677179 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43cbb03b-1a06-4037-9c43-a32817eb272d-istiod-ca-cert\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.677381 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.677227 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nsn4\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-kube-api-access-6nsn4\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.677381 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.677287 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-data\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.677381 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.677313 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-envoy\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.677381 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.677335 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-podinfo\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.677381 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.677358 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-token\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.677617 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.677423 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-credential-socket\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778038 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.777935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-token\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778038 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-credential-socket\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778303 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-socket\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778303 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778107 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-certs\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778303 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778144 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43cbb03b-1a06-4037-9c43-a32817eb272d-istiod-ca-cert\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778303 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nsn4\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-kube-api-access-6nsn4\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778564 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-data\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778568 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-certs\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778603 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-envoy\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778623 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-credential-socket\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778641 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-podinfo\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.778910 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778878 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-data\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.779051 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.778973 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43cbb03b-1a06-4037-9c43-a32817eb272d-istiod-ca-cert\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.779051 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.779012 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-socket\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.780963 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.780942 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-envoy\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.781262 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.781244 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-podinfo\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.788762 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.788741 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nsn4\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-kube-api-access-6nsn4\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.788762 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.788749 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-token\") pod \"e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.808767 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.808746 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:29.955048 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.955018 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq"] Jul 05 13:54:29.957239 ip-10-0-143-21 kubenswrapper[2579]: W0705 13:54:29.957202 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43cbb03b_1a06_4037_9c43_a32817eb272d.slice/crio-96859a3329c4f457c08a45a2b19a7fcbbc58ab226858eaf3023944edc39fe673 WatchSource:0}: Error finding container 96859a3329c4f457c08a45a2b19a7fcbbc58ab226858eaf3023944edc39fe673: Status 404 returned error can't find the container with id 96859a3329c4f457c08a45a2b19a7fcbbc58ab226858eaf3023944edc39fe673 Jul 05 13:54:29.959543 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.959501 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:29.959636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.959587 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:29.959636 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:29.959625 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:30.629439 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:30.629401 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" event={"ID":"43cbb03b-1a06-4037-9c43-a32817eb272d","Type":"ContainerStarted","Data":"ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e"} Jul 05 13:54:30.629850 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:30.629461 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" event={"ID":"43cbb03b-1a06-4037-9c43-a32817eb272d","Type":"ContainerStarted","Data":"96859a3329c4f457c08a45a2b19a7fcbbc58ab226858eaf3023944edc39fe673"} Jul 05 13:54:30.650582 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:30.650528 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" podStartSLOduration=1.650511702 podStartE2EDuration="1.650511702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:54:30.648462359 +0000 UTC m=+2175.276828115" watchObservedRunningTime="2026-07-05 13:54:30.650511702 +0000 UTC m=+2175.278877456" Jul 05 13:54:30.809033 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:30.809001 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:31.814374 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:31.814349 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:32.639221 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:32.639187 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:32.640143 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:32.640122 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:33.865785 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:33.865761 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:34.022278 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022184 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-socket\") pod \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " Jul 05 13:54:34.022278 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022231 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istiod-ca-cert\") pod \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " Jul 05 13:54:34.022509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022307 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-token\") pod \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " Jul 05 13:54:34.022509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022335 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-certs\") pod \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " Jul 05 13:54:34.022509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022371 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-podinfo\") pod \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " Jul 05 13:54:34.022509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022395 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-credential-socket\") pod \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " Jul 05 13:54:34.022509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022424 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgwjx\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-kube-api-access-hgwjx\") pod \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " Jul 05 13:54:34.022509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022446 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-envoy\") pod \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " Jul 05 13:54:34.022509 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022475 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-data\") pod \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\" (UID: \"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9\") " Jul 05 13:54:34.022908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022509 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" (UID: "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:34.022908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022611 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" (UID: "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:34.022908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022711 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" (UID: "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:34.022908 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022754 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" (UID: "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 13:54:34.023119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022915 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:34.023119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022934 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:34.023119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022950 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:34.023119 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.022962 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:34.023476 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.023455 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" (UID: "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:34.024863 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.024836 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" (UID: "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:54:34.024863 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.024848 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" (UID: "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 13:54:34.025228 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.025209 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" (UID: "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 13:54:34.025228 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.025212 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-kube-api-access-hgwjx" (OuterVolumeSpecName: "kube-api-access-hgwjx") pod "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" (UID: "72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9"). InnerVolumeSpecName "kube-api-access-hgwjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 13:54:34.123837 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.123798 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:34.123837 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.123828 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:34.123837 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.123837 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgwjx\" (UniqueName: \"kubernetes.io/projected/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-kube-api-access-hgwjx\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:34.123837 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.123847 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:34.124181 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.123855 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 13:54:34.649406 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.649362 2579 generic.go:356] "Generic (PLEG): container finished" podID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerID="ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1" exitCode=0 Jul 05 13:54:34.649596 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.649423 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" Jul 05 13:54:34.649596 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.649453 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" event={"ID":"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9","Type":"ContainerDied","Data":"ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1"} Jul 05 13:54:34.649596 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.649530 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4" event={"ID":"72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9","Type":"ContainerDied","Data":"9eb3813a3a501fb41e93fef0cc5c9cda0e98649eea6db241c49c36e3bda846a5"} Jul 05 13:54:34.649596 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.649553 2579 scope.go:117] "RemoveContainer" containerID="ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1" Jul 05 13:54:34.658855 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.658830 2579 scope.go:117] "RemoveContainer" containerID="07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0" Jul 05 13:54:34.666821 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.666803 2579 scope.go:117] "RemoveContainer" containerID="ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1" Jul 05 13:54:34.667097 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:54:34.667078 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1\": container with ID starting with ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1 not found: ID does not exist" containerID="ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1" Jul 05 13:54:34.667161 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.667106 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1"} err="failed to get container status \"ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1\": rpc error: code = NotFound desc = could not find container \"ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1\": container with ID starting with ec4e8cbd4ecd453770c24fe3a2ebdf668124fdc0ba17a5e8bb2164df34a028a1 not found: ID does not exist" Jul 05 13:54:34.667161 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.667124 2579 scope.go:117] "RemoveContainer" containerID="07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0" Jul 05 13:54:34.667378 ip-10-0-143-21 kubenswrapper[2579]: E0705 13:54:34.667346 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0\": container with ID starting with 07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0 not found: ID does not exist" containerID="07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0" Jul 05 13:54:34.667421 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.667385 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0"} err="failed to get container status \"07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0\": rpc error: code = NotFound desc = could not find container \"07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0\": container with ID starting with 07137ce872e1266bb482845e778255fcc4e0a0928e392d8fca2f7665ad9adda0 not found: ID does not exist" Jul 05 13:54:34.670491 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.670466 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4"] Jul 05 13:54:34.674781 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:34.674757 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d0bf0c-openshift-default-69559f77f5-bb2h4"] Jul 05 13:54:35.893937 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:35.893902 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" path="/var/lib/kubelet/pods/72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9/volumes" Jul 05 13:54:37.828992 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.828955 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn"] Jul 05 13:54:37.829372 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.829287 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" Jul 05 13:54:37.829372 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.829298 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" Jul 05 13:54:37.829372 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.829371 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" Jul 05 13:54:37.829475 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.829380 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" Jul 05 13:54:37.833859 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.833839 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.836368 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.836350 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-7acb12-openshift-default-dockercfg-qq8sr\"" Jul 05 13:54:37.848881 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.848861 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn"] Jul 05 13:54:37.857620 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.857592 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-socket\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.857764 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.857667 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-podinfo\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.857764 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.857736 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-certs\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.857893 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.857766 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istiod-ca-cert\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.857893 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.857786 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-credential-socket\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.857893 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.857884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-envoy\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.858073 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.857921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-data\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.858073 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.857955 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-token\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.858073 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.857974 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjjpj\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-kube-api-access-pjjpj\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.958982 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.958942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-socket\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959182 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959000 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-podinfo\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959182 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959034 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-certs\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959182 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959078 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istiod-ca-cert\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959182 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959111 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-credential-socket\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959452 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959419 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-socket\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959575 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959423 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-envoy\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959575 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959453 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-credential-socket\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959721 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959578 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-data\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959721 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-token\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959721 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjjpj\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-kube-api-access-pjjpj\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959721 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959473 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-certs\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959842 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istiod-ca-cert\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.959974 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.959854 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-data\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.961780 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.961754 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-envoy\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.961854 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.961753 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-podinfo\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.968197 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.968178 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjjpj\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-kube-api-access-pjjpj\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:37.968300 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:37.968281 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-token\") pod \"e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:38.145671 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:38.145574 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:38.324206 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:38.324020 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn"] Jul 05 13:54:38.331734 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:38.331699 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:38.331810 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:38.331782 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:38.331855 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:38.331821 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:38.669025 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:38.668986 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" event={"ID":"3c441a83-e5af-4bf8-8e42-21b92dd90b04","Type":"ContainerStarted","Data":"2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986"} Jul 05 13:54:38.669217 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:38.669037 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" event={"ID":"3c441a83-e5af-4bf8-8e42-21b92dd90b04","Type":"ContainerStarted","Data":"1a772c819611fd894a2aa05b2565f6b64c329b63d50ae81f8e017494779bbff3"} Jul 05 13:54:38.693540 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:38.693494 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" podStartSLOduration=1.693479008 podStartE2EDuration="1.693479008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 13:54:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 13:54:38.69090668 +0000 UTC m=+2183.319272435" watchObservedRunningTime="2026-07-05 13:54:38.693479008 +0000 UTC m=+2183.321844826" Jul 05 13:54:39.146276 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:39.146229 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:39.151114 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:39.151089 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:39.675298 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:39.675264 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:39.676335 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:39.676313 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:44.701841 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:44.701812 2579 generic.go:356] "Generic (PLEG): container finished" podID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerID="ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e" exitCode=0 Jul 05 13:54:44.702256 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:44.701887 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" event={"ID":"43cbb03b-1a06-4037-9c43-a32817eb272d","Type":"ContainerDied","Data":"ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e"} Jul 05 13:54:44.702299 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:44.702278 2579 scope.go:117] "RemoveContainer" containerID="ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e" Jul 05 13:54:44.703283 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:44.703252 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:44.703382 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:44.703340 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:44.703444 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:44.703387 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:45.712525 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:45.712482 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" event={"ID":"43cbb03b-1a06-4037-9c43-a32817eb272d","Type":"ContainerStarted","Data":"60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68"} Jul 05 13:54:45.809661 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:45.809569 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:45.810789 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:45.810753 2579 patch_prober.go:28] interesting pod/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 05 13:54:45.810941 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:45.810820 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 05 13:54:46.809087 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:46.809050 2579 patch_prober.go:28] interesting pod/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 05 13:54:46.809729 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:46.809699 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 05 13:54:47.639976 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:47.639933 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:47.809658 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:47.809621 2579 patch_prober.go:28] interesting pod/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 05 13:54:47.810280 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:47.810247 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 05 13:54:48.809406 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:48.809360 2579 patch_prober.go:28] interesting pod/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 05 13:54:48.809619 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:48.809430 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 05 13:54:49.820741 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:49.820704 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:49.822073 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:49.822045 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 13:54:55.503176 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:55.501891 2579 patch_prober.go:28] interesting pod/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:51840->10.132.0.45:15021: read: connection reset by peer" start-of-body= Jul 05 13:54:55.503176 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:55.501982 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:51840->10.132.0.45:15021: read: connection reset by peer" Jul 05 13:54:55.769351 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:55.769259 2579 generic.go:356] "Generic (PLEG): container finished" podID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerID="2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986" exitCode=0 Jul 05 13:54:55.769351 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:55.769294 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" event={"ID":"3c441a83-e5af-4bf8-8e42-21b92dd90b04","Type":"ContainerDied","Data":"2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986"} Jul 05 13:54:55.769707 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:55.769669 2579 scope.go:117] "RemoveContainer" containerID="2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986" Jul 05 13:54:55.770728 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:55.770692 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:55.770780 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:55.770769 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:55.770820 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:55.770802 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 05 13:54:56.776065 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:56.776033 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" event={"ID":"3c441a83-e5af-4bf8-8e42-21b92dd90b04","Type":"ContainerStarted","Data":"d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e"} Jul 05 13:54:57.146060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:57.146022 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:54:57.146269 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:57.146247 2579 patch_prober.go:28] interesting pod/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 05 13:54:57.146320 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:57.146297 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 05 13:54:58.146060 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:58.146021 2579 patch_prober.go:28] interesting pod/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 05 13:54:58.146518 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:58.146087 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 05 13:54:59.146152 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:59.146120 2579 patch_prober.go:28] interesting pod/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 05 13:54:59.146534 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:54:59.146172 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 05 13:55:00.146426 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:55:00.146396 2579 patch_prober.go:28] interesting pod/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 05 13:55:00.146855 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:55:00.146448 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 05 13:55:01.149501 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:55:01.149468 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:55:01.149967 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:55:01.149741 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 13:55:01.150521 ip-10-0-143-21 kubenswrapper[2579]: I0705 13:55:01.150506 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 14:01:25.622970 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:25.622931 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq"] Jul 05 14:01:25.623502 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:25.623189 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerName="istio-proxy" containerID="cri-o://60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68" gracePeriod=30 Jul 05 14:01:30.876035 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.876009 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 14:01:30.945109 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945039 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-envoy\") pod \"43cbb03b-1a06-4037-9c43-a32817eb272d\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " Jul 05 14:01:30.945109 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945077 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-podinfo\") pod \"43cbb03b-1a06-4037-9c43-a32817eb272d\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " Jul 05 14:01:30.945109 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945109 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43cbb03b-1a06-4037-9c43-a32817eb272d-istiod-ca-cert\") pod \"43cbb03b-1a06-4037-9c43-a32817eb272d\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " Jul 05 14:01:30.945373 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945125 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-data\") pod \"43cbb03b-1a06-4037-9c43-a32817eb272d\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " Jul 05 14:01:30.945373 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945153 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nsn4\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-kube-api-access-6nsn4\") pod \"43cbb03b-1a06-4037-9c43-a32817eb272d\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " Jul 05 14:01:30.945373 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945189 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-socket\") pod \"43cbb03b-1a06-4037-9c43-a32817eb272d\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " Jul 05 14:01:30.945373 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945236 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-certs\") pod \"43cbb03b-1a06-4037-9c43-a32817eb272d\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " Jul 05 14:01:30.945373 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945278 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-token\") pod \"43cbb03b-1a06-4037-9c43-a32817eb272d\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " Jul 05 14:01:30.945373 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945307 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-credential-socket\") pod \"43cbb03b-1a06-4037-9c43-a32817eb272d\" (UID: \"43cbb03b-1a06-4037-9c43-a32817eb272d\") " Jul 05 14:01:30.945708 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945524 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "43cbb03b-1a06-4037-9c43-a32817eb272d" (UID: "43cbb03b-1a06-4037-9c43-a32817eb272d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:30.945708 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945556 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43cbb03b-1a06-4037-9c43-a32817eb272d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "43cbb03b-1a06-4037-9c43-a32817eb272d" (UID: "43cbb03b-1a06-4037-9c43-a32817eb272d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:01:30.946001 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945735 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "43cbb03b-1a06-4037-9c43-a32817eb272d" (UID: "43cbb03b-1a06-4037-9c43-a32817eb272d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:30.946001 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.945777 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "43cbb03b-1a06-4037-9c43-a32817eb272d" (UID: "43cbb03b-1a06-4037-9c43-a32817eb272d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:30.946113 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.946005 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-data" (OuterVolumeSpecName: "istio-data") pod "43cbb03b-1a06-4037-9c43-a32817eb272d" (UID: "43cbb03b-1a06-4037-9c43-a32817eb272d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:30.947404 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.947375 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "43cbb03b-1a06-4037-9c43-a32817eb272d" (UID: "43cbb03b-1a06-4037-9c43-a32817eb272d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:30.947528 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.947466 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-kube-api-access-6nsn4" (OuterVolumeSpecName: "kube-api-access-6nsn4") pod "43cbb03b-1a06-4037-9c43-a32817eb272d" (UID: "43cbb03b-1a06-4037-9c43-a32817eb272d"). InnerVolumeSpecName "kube-api-access-6nsn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:01:30.947528 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.947516 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "43cbb03b-1a06-4037-9c43-a32817eb272d" (UID: "43cbb03b-1a06-4037-9c43-a32817eb272d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 14:01:30.947779 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:30.947753 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-token" (OuterVolumeSpecName: "istio-token") pod "43cbb03b-1a06-4037-9c43-a32817eb272d" (UID: "43cbb03b-1a06-4037-9c43-a32817eb272d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:01:31.046248 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.046206 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:31.046248 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.046244 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:31.046248 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.046257 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:31.046484 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.046269 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:31.046484 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.046281 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:31.046484 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.046293 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43cbb03b-1a06-4037-9c43-a32817eb272d-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:31.046484 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.046302 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:31.046484 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.046312 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nsn4\" (UniqueName: \"kubernetes.io/projected/43cbb03b-1a06-4037-9c43-a32817eb272d-kube-api-access-6nsn4\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:31.046484 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.046323 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43cbb03b-1a06-4037-9c43-a32817eb272d-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:31.618266 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.618228 2579 generic.go:356] "Generic (PLEG): container finished" podID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerID="60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68" exitCode=0 Jul 05 14:01:31.618564 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.618271 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" Jul 05 14:01:31.618564 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.618289 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" event={"ID":"43cbb03b-1a06-4037-9c43-a32817eb272d","Type":"ContainerDied","Data":"60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68"} Jul 05 14:01:31.618564 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.618338 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq" event={"ID":"43cbb03b-1a06-4037-9c43-a32817eb272d","Type":"ContainerDied","Data":"96859a3329c4f457c08a45a2b19a7fcbbc58ab226858eaf3023944edc39fe673"} Jul 05 14:01:31.618564 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.618362 2579 scope.go:117] "RemoveContainer" containerID="60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68" Jul 05 14:01:31.630262 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.630241 2579 scope.go:117] "RemoveContainer" containerID="ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e" Jul 05 14:01:31.638598 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.638576 2579 scope.go:117] "RemoveContainer" containerID="60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68" Jul 05 14:01:31.638865 ip-10-0-143-21 kubenswrapper[2579]: E0705 14:01:31.638846 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68\": container with ID starting with 60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68 not found: ID does not exist" containerID="60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68" Jul 05 14:01:31.638939 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.638874 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68"} err="failed to get container status \"60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68\": rpc error: code = NotFound desc = could not find container \"60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68\": container with ID starting with 60625df780080ec85de1ba8e3d86aad35f8535db18054602a7a4e39b71d9fc68 not found: ID does not exist" Jul 05 14:01:31.638939 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.638891 2579 scope.go:117] "RemoveContainer" containerID="ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e" Jul 05 14:01:31.639130 ip-10-0-143-21 kubenswrapper[2579]: E0705 14:01:31.639105 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e\": container with ID starting with ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e not found: ID does not exist" containerID="ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e" Jul 05 14:01:31.639174 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.639139 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e"} err="failed to get container status \"ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e\": rpc error: code = NotFound desc = could not find container \"ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e\": container with ID starting with ad1b40c9f732ebf7be7275123ec27561d4098716be80cad4d7b43b50cb02720e not found: ID does not exist" Jul 05 14:01:31.640657 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.640639 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq"] Jul 05 14:01:31.644425 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.644405 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c2f889-openshift-default-7575cd98d7-d2sgq"] Jul 05 14:01:31.893667 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:31.893583 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" path="/var/lib/kubelet/pods/43cbb03b-1a06-4037-9c43-a32817eb272d/volumes" Jul 05 14:01:32.626768 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:32.626735 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn"] Jul 05 14:01:32.626957 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:32.626936 2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" containerID="cri-o://d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e" gracePeriod=30 Jul 05 14:01:37.874822 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:37.874798 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 14:01:38.007597 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007515 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istiod-ca-cert\") pod \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " Jul 05 14:01:38.007597 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007551 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-credential-socket\") pod \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " Jul 05 14:01:38.007597 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007570 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-envoy\") pod \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " Jul 05 14:01:38.007597 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007592 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjjpj\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-kube-api-access-pjjpj\") pod \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " Jul 05 14:01:38.007967 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007627 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-token\") pod \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " Jul 05 14:01:38.007967 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007699 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-certs\") pod \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " Jul 05 14:01:38.007967 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007793 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-podinfo\") pod \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " Jul 05 14:01:38.007967 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007826 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-data\") pod \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " Jul 05 14:01:38.007967 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007858 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-socket\") pod \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\" (UID: \"3c441a83-e5af-4bf8-8e42-21b92dd90b04\") " Jul 05 14:01:38.007967 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.007870 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c441a83-e5af-4bf8-8e42-21b92dd90b04" (UID: "3c441a83-e5af-4bf8-8e42-21b92dd90b04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:38.008276 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.008050 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c441a83-e5af-4bf8-8e42-21b92dd90b04" (UID: "3c441a83-e5af-4bf8-8e42-21b92dd90b04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 14:01:38.008276 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.008225 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c441a83-e5af-4bf8-8e42-21b92dd90b04" (UID: "3c441a83-e5af-4bf8-8e42-21b92dd90b04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:38.008430 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.008297 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-credential-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:38.008430 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.008317 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istiod-ca-cert\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:38.008430 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.008320 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c441a83-e5af-4bf8-8e42-21b92dd90b04" (UID: "3c441a83-e5af-4bf8-8e42-21b92dd90b04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:38.008711 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.008672 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c441a83-e5af-4bf8-8e42-21b92dd90b04" (UID: "3c441a83-e5af-4bf8-8e42-21b92dd90b04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:38.010146 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.010112 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c441a83-e5af-4bf8-8e42-21b92dd90b04" (UID: "3c441a83-e5af-4bf8-8e42-21b92dd90b04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 05 14:01:38.010146 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.010124 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-kube-api-access-pjjpj" (OuterVolumeSpecName: "kube-api-access-pjjpj") pod "3c441a83-e5af-4bf8-8e42-21b92dd90b04" (UID: "3c441a83-e5af-4bf8-8e42-21b92dd90b04"). InnerVolumeSpecName "kube-api-access-pjjpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:01:38.010417 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.010395 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c441a83-e5af-4bf8-8e42-21b92dd90b04" (UID: "3c441a83-e5af-4bf8-8e42-21b92dd90b04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 14:01:38.010417 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.010398 2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c441a83-e5af-4bf8-8e42-21b92dd90b04" (UID: "3c441a83-e5af-4bf8-8e42-21b92dd90b04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 14:01:38.109387 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.109354 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-podinfo\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:38.109387 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.109380 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-data\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:38.109387 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.109393 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-socket\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:38.109601 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.109404 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-envoy\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:38.109601 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.109417 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjjpj\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-kube-api-access-pjjpj\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:38.109601 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.109429 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c441a83-e5af-4bf8-8e42-21b92dd90b04-istio-token\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:38.109601 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.109440 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c441a83-e5af-4bf8-8e42-21b92dd90b04-workload-certs\") on node \"ip-10-0-143-21.ec2.internal\" DevicePath \"\"" Jul 05 14:01:38.481119 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.481091 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720985-hwbsq_f51cf3a2-96c0-42c9-a809-6133e4df2c88/cleanup/0.log" Jul 05 14:01:38.655956 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.655921 2579 generic.go:356] "Generic (PLEG): container finished" podID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerID="d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e" exitCode=0 Jul 05 14:01:38.656149 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.655975 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" Jul 05 14:01:38.656149 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.656008 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" event={"ID":"3c441a83-e5af-4bf8-8e42-21b92dd90b04","Type":"ContainerDied","Data":"d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e"} Jul 05 14:01:38.656149 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.656065 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn" event={"ID":"3c441a83-e5af-4bf8-8e42-21b92dd90b04","Type":"ContainerDied","Data":"1a772c819611fd894a2aa05b2565f6b64c329b63d50ae81f8e017494779bbff3"} Jul 05 14:01:38.656149 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.656088 2579 scope.go:117] "RemoveContainer" containerID="d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e" Jul 05 14:01:38.665290 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.665274 2579 scope.go:117] "RemoveContainer" containerID="2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986" Jul 05 14:01:38.673114 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.673098 2579 scope.go:117] "RemoveContainer" containerID="d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e" Jul 05 14:01:38.673344 ip-10-0-143-21 kubenswrapper[2579]: E0705 14:01:38.673326 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e\": container with ID starting with d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e not found: ID does not exist" containerID="d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e" Jul 05 14:01:38.673397 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.673353 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e"} err="failed to get container status \"d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e\": rpc error: code = NotFound desc = could not find container \"d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e\": container with ID starting with d9d338fcec4247619cb85c3db72189f960575d6a288b27386d44ef935cf1ae9e not found: ID does not exist" Jul 05 14:01:38.673397 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.673371 2579 scope.go:117] "RemoveContainer" containerID="2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986" Jul 05 14:01:38.673616 ip-10-0-143-21 kubenswrapper[2579]: E0705 14:01:38.673599 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986\": container with ID starting with 2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986 not found: ID does not exist" containerID="2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986" Jul 05 14:01:38.673664 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.673622 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986"} err="failed to get container status \"2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986\": rpc error: code = NotFound desc = could not find container \"2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986\": container with ID starting with 2d0ffe4eda41833942489a98cbe6284e9985527367c8eaff6e52592bdf06f986 not found: ID does not exist" Jul 05 14:01:38.678024 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.678002 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn"] Jul 05 14:01:38.681801 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:38.681778 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-7acb12-openshift-default-74995cdff5-vlmcn"] Jul 05 14:01:39.293336 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:39.293311 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54547f4bcb-qhsrc_7217a8a5-d941-4a4d-8e55-bedac65d519a/copy-manifests/0.log" Jul 05 14:01:39.299806 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:39.299781 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54547f4bcb-qhsrc_7217a8a5-d941-4a4d-8e55-bedac65d519a/manager/0.log" Jul 05 14:01:39.408370 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:39.408339 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-pljv9_6317a779-54ed-46cc-94bd-77a561f3a5f8/postgres/0.log" Jul 05 14:01:39.894043 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:39.894010 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" path="/var/lib/kubelet/pods/3c441a83-e5af-4bf8-8e42-21b92dd90b04/volumes" Jul 05 14:01:41.890695 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:41.890645 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dgls9_f91e152d-e714-45f0-9cd3-fdad49637936/istio-proxy/0.log" Jul 05 14:01:42.010102 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:42.010069 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-69mw4_dbe8e3cf-3126-4cad-880c-313c51461d20/discovery/0.log" Jul 05 14:01:42.349673 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:42.349642 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fzz4h_490ff508-add3-40b5-bc75-6a3f69acd8c6/istio-proxy/1.log" Jul 05 14:01:43.247253 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:43.247221 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-xvlt7_5dc85d56-9c91-42f2-9048-708098ed16cc/main/0.log" Jul 05 14:01:43.479578 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:43.479541 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-crq56_4e3dfe9b-104d-4a61-9766-205643d58666/main/0.log" Jul 05 14:01:50.863731 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:50.863669 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h4vlk_dcb90eb9-329e-4a35-9ced-6d86eb35eb4f/global-pull-secret-syncer/0.log" Jul 05 14:01:51.019311 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:51.019281 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h9dvp_cb9ca114-8751-44b3-b13a-713cf3f44f18/konnectivity-agent/0.log" Jul 05 14:01:51.144910 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:51.144820 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-21.ec2.internal_33d35a027521fec9443ee23f19c336c8/haproxy/0.log" Jul 05 14:01:58.448924 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.448806 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-p95c6_c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb/kube-state-metrics/0.log" Jul 05 14:01:58.469574 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.469550 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-p95c6_c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb/kube-rbac-proxy-main/0.log" Jul 05 14:01:58.490919 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.490892 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-p95c6_c5087350-cb82-4fb3-8bdb-4ea64a0f0bfb/kube-rbac-proxy-self/0.log" Jul 05 14:01:58.705369 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.705288 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jml4c_dc45372b-1060-4080-a52b-de78d72d2055/node-exporter/0.log" Jul 05 14:01:58.733958 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.733932 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jml4c_dc45372b-1060-4080-a52b-de78d72d2055/kube-rbac-proxy/0.log" Jul 05 14:01:58.761930 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.761901 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jml4c_dc45372b-1060-4080-a52b-de78d72d2055/init-textfile/0.log" Jul 05 14:01:58.904480 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.904452 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sjnf2_aa7e5237-b83e-47d9-b627-bbd790997257/kube-rbac-proxy-main/0.log" Jul 05 14:01:58.932359 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.932336 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sjnf2_aa7e5237-b83e-47d9-b627-bbd790997257/kube-rbac-proxy-self/0.log" Jul 05 14:01:58.957973 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.957902 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-sjnf2_aa7e5237-b83e-47d9-b627-bbd790997257/openshift-state-metrics/0.log" Jul 05 14:01:58.996578 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:58.996549 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e74bc18-36b5-476f-b2f7-411b94eae2f6/prometheus/0.log" Jul 05 14:01:59.017015 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:59.016993 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e74bc18-36b5-476f-b2f7-411b94eae2f6/config-reloader/0.log" Jul 05 14:01:59.047280 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:59.047255 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e74bc18-36b5-476f-b2f7-411b94eae2f6/thanos-sidecar/0.log" Jul 05 14:01:59.071878 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:59.071851 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e74bc18-36b5-476f-b2f7-411b94eae2f6/kube-rbac-proxy-web/0.log" Jul 05 14:01:59.096718 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:59.096666 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e74bc18-36b5-476f-b2f7-411b94eae2f6/kube-rbac-proxy/0.log" Jul 05 14:01:59.118801 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:59.118782 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e74bc18-36b5-476f-b2f7-411b94eae2f6/kube-rbac-proxy-thanos/0.log" Jul 05 14:01:59.141437 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:01:59.141412 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0e74bc18-36b5-476f-b2f7-411b94eae2f6/init-config-reloader/0.log" Jul 05 14:02:00.720537 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:00.720508 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-z2m2z_ccebd28f-1a0e-4f00-9447-c005b8122899/networking-console-plugin/0.log" Jul 05 14:02:01.962884 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.962851 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b"] Jul 05 14:02:01.963264 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963172 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerName="istio-proxy" Jul 05 14:02:01.963264 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963184 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerName="istio-proxy" Jul 05 14:02:01.963264 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963192 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" Jul 05 14:02:01.963264 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963198 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" Jul 05 14:02:01.963264 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963207 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" Jul 05 14:02:01.963264 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963212 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="72fd3dd0-4fb3-4456-9674-7e2d2fdfb9c9" containerName="istio-proxy" Jul 05 14:02:01.963456 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963279 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerName="istio-proxy" Jul 05 14:02:01.963456 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963286 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="43cbb03b-1a06-4037-9c43-a32817eb272d" containerName="istio-proxy" Jul 05 14:02:01.963456 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963292 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" Jul 05 14:02:01.963456 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.963299 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c441a83-e5af-4bf8-8e42-21b92dd90b04" containerName="istio-proxy" Jul 05 14:02:01.966517 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.966487 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:01.966671 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.966579 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7446c94d6f-rshlr_16249bc8-bf8f-4389-895c-5ea504981578/console/0.log" Jul 05 14:02:01.968745 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.968726 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-srwzm\"/\"kube-root-ca.crt\"" Jul 05 14:02:01.969639 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.969622 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-srwzm\"/\"openshift-service-ca.crt\"" Jul 05 14:02:01.969745 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.969622 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-srwzm\"/\"default-dockercfg-hwlnw\"" Jul 05 14:02:01.976415 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:01.976396 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b"] Jul 05 14:02:02.121473 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.121440 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-sys\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.121663 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.121489 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-proc\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.121663 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.121514 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9k5w\" (UniqueName: \"kubernetes.io/projected/acdcbcf9-f622-42a4-a38c-4718b69c30a4-kube-api-access-f9k5w\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.121663 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.121569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-lib-modules\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.121663 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.121621 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-podres\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.222486 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.222412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-sys\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.222486 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.222464 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-proc\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.222486 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.222481 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9k5w\" (UniqueName: \"kubernetes.io/projected/acdcbcf9-f622-42a4-a38c-4718b69c30a4-kube-api-access-f9k5w\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.222713 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.222514 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-lib-modules\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.222713 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.222525 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-sys\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.222713 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.222577 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-proc\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.222713 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.222587 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-podres\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.222713 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.222644 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-lib-modules\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.222879 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.222729 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/acdcbcf9-f622-42a4-a38c-4718b69c30a4-podres\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.230092 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.230070 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9k5w\" (UniqueName: \"kubernetes.io/projected/acdcbcf9-f622-42a4-a38c-4718b69c30a4-kube-api-access-f9k5w\") pod \"perf-node-gather-daemonset-smm5b\" (UID: \"acdcbcf9-f622-42a4-a38c-4718b69c30a4\") " pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.277335 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.277308 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.413515 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.413484 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b"] Jul 05 14:02:02.417166 ip-10-0-143-21 kubenswrapper[2579]: W0705 14:02:02.417141 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podacdcbcf9_f622_42a4_a38c_4718b69c30a4.slice/crio-e074379d53258d9f6c3b32c346fc99a615da7730eb1e66238614e4aad84eda53 WatchSource:0}: Error finding container e074379d53258d9f6c3b32c346fc99a615da7730eb1e66238614e4aad84eda53: Status 404 returned error can't find the container with id e074379d53258d9f6c3b32c346fc99a615da7730eb1e66238614e4aad84eda53 Jul 05 14:02:02.768979 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.768941 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" event={"ID":"acdcbcf9-f622-42a4-a38c-4718b69c30a4","Type":"ContainerStarted","Data":"baffbb065d03a881eb037c9deecee0a0b10752fc413ed191e55184944bb120bd"} Jul 05 14:02:02.769156 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.768994 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" event={"ID":"acdcbcf9-f622-42a4-a38c-4718b69c30a4","Type":"ContainerStarted","Data":"e074379d53258d9f6c3b32c346fc99a615da7730eb1e66238614e4aad84eda53"} Jul 05 14:02:02.769156 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.769026 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:02.786588 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:02.786537 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" podStartSLOduration=1.7865221500000001 podStartE2EDuration="1.78652215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 14:02:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 14:02:02.784895858 +0000 UTC m=+2627.413261617" watchObservedRunningTime="2026-07-05 14:02:02.78652215 +0000 UTC m=+2627.414887904" Jul 05 14:02:03.845743 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:03.845714 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jgqxl_60aa1aed-d099-4605-b73f-602f56a82c38/dns/0.log" Jul 05 14:02:03.867822 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:03.867793 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jgqxl_60aa1aed-d099-4605-b73f-602f56a82c38/kube-rbac-proxy/0.log" Jul 05 14:02:03.947158 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:03.947125 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zpbbv_3f358ef2-7b32-47c0-aaac-5c1048a34441/dns-node-resolver/0.log" Jul 05 14:02:04.587639 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:04.587606 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sk68q_373998a7-e4cd-48b0-96fa-a4962a0fbe29/node-ca/0.log" Jul 05 14:02:05.520497 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:05.520465 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dgls9_f91e152d-e714-45f0-9cd3-fdad49637936/istio-proxy/0.log" Jul 05 14:02:05.723094 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:05.723063 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-69mw4_dbe8e3cf-3126-4cad-880c-313c51461d20/discovery/0.log" Jul 05 14:02:05.806302 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:05.806276 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fzz4h_490ff508-add3-40b5-bc75-6a3f69acd8c6/istio-proxy/0.log" Jul 05 14:02:05.863808 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:05.863765 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fzz4h_490ff508-add3-40b5-bc75-6a3f69acd8c6/istio-proxy/1.log" Jul 05 14:02:06.666779 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:06.666740 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7ppkp_5b0d1007-eefb-4006-8830-9f5bd22c83a3/serve-healthcheck-canary/0.log" Jul 05 14:02:07.178001 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:07.177963 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-xbxlt_7cf49274-f63e-4e52-a228-1a1e8e1e6ac1/insights-operator/0.log" Jul 05 14:02:07.178907 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:07.178882 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-xbxlt_7cf49274-f63e-4e52-a228-1a1e8e1e6ac1/insights-operator/1.log" Jul 05 14:02:07.204458 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:07.204435 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cqxt_42b6e001-bc2d-48a0-b007-2a1154128b43/kube-rbac-proxy/0.log" Jul 05 14:02:07.228856 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:07.228832 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cqxt_42b6e001-bc2d-48a0-b007-2a1154128b43/exporter/0.log" Jul 05 14:02:07.252735 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:07.252708 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cqxt_42b6e001-bc2d-48a0-b007-2a1154128b43/extractor/0.log" Jul 05 14:02:08.783594 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:08.783568 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-srwzm/perf-node-gather-daemonset-smm5b" Jul 05 14:02:09.901293 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:09.901268 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29720985-hwbsq_f51cf3a2-96c0-42c9-a809-6133e4df2c88/cleanup/0.log" Jul 05 14:02:10.328215 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:10.328184 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54547f4bcb-qhsrc_7217a8a5-d941-4a4d-8e55-bedac65d519a/manager/0.log" Jul 05 14:02:10.351439 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:10.351414 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54547f4bcb-qhsrc_7217a8a5-d941-4a4d-8e55-bedac65d519a/copy-manifests/0.log" Jul 05 14:02:10.380255 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:10.380229 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-pljv9_6317a779-54ed-46cc-94bd-77a561f3a5f8/postgres/0.log" Jul 05 14:02:11.827953 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:11.827914 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-cfb4f694f-6hvsd_95ab55db-db4f-4c88-90df-3840cc7d5727/manager/0.log" Jul 05 14:02:12.581470 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:12.581435 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dgls9_f91e152d-e714-45f0-9cd3-fdad49637936/istio-proxy/0.log" Jul 05 14:02:12.772998 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:12.772918 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-69mw4_dbe8e3cf-3126-4cad-880c-313c51461d20/discovery/0.log" Jul 05 14:02:12.826386 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:12.826361 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fzz4h_490ff508-add3-40b5-bc75-6a3f69acd8c6/istio-proxy/0.log" Jul 05 14:02:12.880905 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:12.880873 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fzz4h_490ff508-add3-40b5-bc75-6a3f69acd8c6/istio-proxy/1.log" Jul 05 14:02:14.099354 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:14.099318 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dgls9_f91e152d-e714-45f0-9cd3-fdad49637936/istio-proxy/0.log" Jul 05 14:02:14.312532 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:14.312498 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-69mw4_dbe8e3cf-3126-4cad-880c-313c51461d20/discovery/0.log" Jul 05 14:02:14.366792 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:14.366710 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fzz4h_490ff508-add3-40b5-bc75-6a3f69acd8c6/istio-proxy/0.log" Jul 05 14:02:14.425807 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:14.425779 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-fzz4h_490ff508-add3-40b5-bc75-6a3f69acd8c6/istio-proxy/1.log" Jul 05 14:02:18.502345 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:18.502316 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fr86_3e7cb017-f290-456f-8c62-a896ceb8e01e/kube-multus-additional-cni-plugins/0.log" Jul 05 14:02:18.523932 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:18.523910 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fr86_3e7cb017-f290-456f-8c62-a896ceb8e01e/egress-router-binary-copy/0.log" Jul 05 14:02:18.545412 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:18.545387 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fr86_3e7cb017-f290-456f-8c62-a896ceb8e01e/cni-plugins/0.log" Jul 05 14:02:18.567660 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:18.567639 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fr86_3e7cb017-f290-456f-8c62-a896ceb8e01e/bond-cni-plugin/0.log" Jul 05 14:02:18.591206 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:18.591188 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fr86_3e7cb017-f290-456f-8c62-a896ceb8e01e/routeoverride-cni/0.log" Jul 05 14:02:18.614501 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:18.614483 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fr86_3e7cb017-f290-456f-8c62-a896ceb8e01e/whereabouts-cni-bincopy/0.log" Jul 05 14:02:18.635796 ip-10-0-143-21 kubenswrapper[2579]: I0705 14:02:18.635767 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2fr86_3e7cb017-f290-456f-8c62-a896ceb8e01e/whereabouts-cni/0.log"