Jun 02 20:02:03.662283 ip-10-0-134-118 systemd[1]: Starting Kubernetes Kubelet... Jun 02 20:02:04.177044 ip-10-0-134-118 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 20:02:04.177044 ip-10-0-134-118 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 20:02:04.177044 ip-10-0-134-118 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 20:02:04.177044 ip-10-0-134-118 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 20:02:04.177044 ip-10-0-134-118 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 20:02:04.179929 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.179751 2577 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 20:02:04.182590 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182574 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 20:02:04.182590 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182590 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182594 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182599 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182602 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182605 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182608 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182611 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182614 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182617 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182620 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 20:02:04.182656 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.182623 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 20:02:04.184436 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184425 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 20:02:04.184436 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184434 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 20:02:04.184436 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184438 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184442 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184445 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184448 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184451 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184454 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184457 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184460 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184462 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184466 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184469 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184471 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184474 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184476 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184479 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184482 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184484 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184487 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184490 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184492 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 20:02:04.184525 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184495 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184497 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184500 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184502 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184505 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184507 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184509 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184512 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184514 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184517 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184520 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184522 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184525 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184528 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184531 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184533 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184536 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184538 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184542 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 20:02:04.185015 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184544 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184547 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184549 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184552 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184554 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184557 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184560 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184562 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184565 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184567 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184569 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184572 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184574 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184577 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184579 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184584 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184589 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184591 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184594 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184597 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 20:02:04.185517 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184599 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184602 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184605 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184608 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184611 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184614 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184617 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184621 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184624 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184627 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184630 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184632 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.184635 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185040 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185046 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185049 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185052 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185055 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185057 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 20:02:04.186016 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185060 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185063 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185065 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185068 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185071 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185074 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185076 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185079 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185083 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185086 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185089 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185091 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185094 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185096 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185099 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185102 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185105 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185108 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185110 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185113 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 20:02:04.186473 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185116 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185118 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185121 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185123 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185126 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185129 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185132 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185134 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185136 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185139 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185141 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185144 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185147 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185149 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185152 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185155 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185157 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185159 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185162 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185164 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 20:02:04.187021 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185167 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185170 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185172 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185175 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185177 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185180 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185182 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185185 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185188 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185191 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185194 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185196 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185199 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185202 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185205 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185208 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185210 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185214 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185216 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185218 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 20:02:04.187523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185221 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185223 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185226 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185228 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185231 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185233 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185237 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185241 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185244 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185247 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185250 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185253 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185256 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185258 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185261 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185263 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185265 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185268 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185270 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 20:02:04.188038 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.185273 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186068 2577 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186080 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186088 2577 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186092 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186097 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186100 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186104 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186109 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186112 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186116 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186119 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186123 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186126 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186129 2577 flags.go:64] FLAG: --cgroup-root="" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186132 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186135 2577 flags.go:64] FLAG: --client-ca-file="" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186138 2577 flags.go:64] FLAG: --cloud-config="" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186141 2577 flags.go:64] FLAG: --cloud-provider="external" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186144 2577 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186148 2577 flags.go:64] FLAG: --cluster-domain="" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186151 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186154 2577 flags.go:64] FLAG: --config-dir="" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186157 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186161 2577 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 20:02:04.188515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186165 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186167 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186170 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186174 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186177 2577 flags.go:64] FLAG: --contention-profiling="false" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186180 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186183 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186186 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186189 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186193 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186200 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186203 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186206 2577 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186209 2577 flags.go:64] FLAG: --enable-server="true" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186212 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186218 2577 flags.go:64] FLAG: --event-burst="100" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186221 2577 flags.go:64] FLAG: --event-qps="50" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186224 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186227 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186230 2577 flags.go:64] FLAG: --eviction-hard="" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186234 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186237 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186240 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186243 2577 flags.go:64] FLAG: --eviction-soft="" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186246 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 20:02:04.189137 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186249 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186252 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186255 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186258 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186260 2577 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186264 2577 flags.go:64] FLAG: --feature-gates="" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186268 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186271 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186274 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186277 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186280 2577 flags.go:64] FLAG: --healthz-port="10248" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186283 2577 flags.go:64] FLAG: --help="false" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186286 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186289 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186292 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186295 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186299 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186302 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186306 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186308 2577 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186311 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186314 2577 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186317 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186320 2577 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 20:02:04.189743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186323 2577 flags.go:64] FLAG: --kube-reserved="" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186325 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186328 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186331 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186334 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186337 2577 flags.go:64] FLAG: --lock-file="" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186340 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186343 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186345 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186351 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186353 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186356 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186360 2577 flags.go:64] FLAG: --logging-format="text" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186363 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186366 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186369 2577 flags.go:64] FLAG: --manifest-url="" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186372 2577 flags.go:64] FLAG: --manifest-url-header="" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186376 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186379 2577 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186384 2577 flags.go:64] FLAG: --max-pods="110" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186386 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186389 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186393 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186396 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186399 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 20:02:04.190342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186403 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186406 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186414 2577 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186417 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186420 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186424 2577 flags.go:64] FLAG: --pod-cidr="" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186427 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186432 2577 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186435 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186439 2577 flags.go:64] FLAG: --pods-per-core="0" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186442 2577 flags.go:64] FLAG: --port="10250" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186445 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186447 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02fb9a003244e4ccc" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186451 2577 flags.go:64] FLAG: --qos-reserved="" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186454 2577 flags.go:64] FLAG: --read-only-port="10255" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186456 2577 flags.go:64] FLAG: --register-node="true" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186459 2577 flags.go:64] FLAG: --register-schedulable="true" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186462 2577 flags.go:64] FLAG: --register-with-taints="" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186466 2577 flags.go:64] FLAG: --registry-burst="10" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186469 2577 flags.go:64] FLAG: --registry-qps="5" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186471 2577 flags.go:64] FLAG: --reserved-cpus="" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186474 2577 flags.go:64] FLAG: --reserved-memory="" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186478 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186481 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186484 2577 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 20:02:04.190982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186487 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186490 2577 flags.go:64] FLAG: --runonce="false" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186492 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186495 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186498 2577 flags.go:64] FLAG: --seccomp-default="false" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186501 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186504 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186507 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186510 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186514 2577 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186517 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186520 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186523 2577 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186529 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186532 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186535 2577 flags.go:64] FLAG: --system-cgroups="" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186538 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186544 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186546 2577 flags.go:64] FLAG: --tls-cert-file="" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186549 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186553 2577 flags.go:64] FLAG: --tls-min-version="" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186556 2577 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186559 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186562 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186565 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 20:02:04.191611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186568 2577 flags.go:64] FLAG: --v="2" Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186572 2577 flags.go:64] FLAG: --version="false" Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186576 2577 flags.go:64] FLAG: --vmodule="" Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186580 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186584 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186677 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186681 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186684 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186687 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186689 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186692 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186695 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186700 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186702 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186705 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186707 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186711 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186714 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186717 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186720 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 20:02:04.192303 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186725 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186728 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186731 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186733 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186736 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186739 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186741 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186743 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186748 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186751 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186754 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186756 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186759 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186762 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186764 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186767 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186769 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186772 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186774 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186777 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 20:02:04.192789 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186779 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186782 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186784 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186787 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186791 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186794 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186797 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186799 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186802 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186805 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186807 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186811 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186815 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186818 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186821 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186823 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186826 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186828 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186831 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 20:02:04.193316 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186834 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186836 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186839 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186841 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186844 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186846 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186849 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186851 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186854 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186857 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186859 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186862 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186864 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186867 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186870 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186872 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186874 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186881 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186884 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186886 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 20:02:04.193784 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186889 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186892 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186894 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186896 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186899 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186903 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186906 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186909 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186911 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186914 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186916 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.186919 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.186925 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.193495 2577 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.193510 2577 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 20:02:04.194308 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193579 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193585 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193588 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193592 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193595 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193598 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193601 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193604 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193607 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193610 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193612 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193615 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193619 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193624 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193627 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193629 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193632 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193635 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193638 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193641 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 20:02:04.194694 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193644 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193646 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193650 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193653 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193655 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193658 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193660 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193663 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193666 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193668 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193671 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193674 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193677 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193679 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193682 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193686 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193688 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193691 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193694 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193696 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 20:02:04.195228 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193699 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193701 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193704 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193707 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193709 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193712 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193714 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193716 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193719 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193721 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193724 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193726 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193729 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193731 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193734 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193737 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193740 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193742 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193745 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193747 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 20:02:04.195716 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193750 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193752 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193754 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193758 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193760 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193763 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193766 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193769 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193771 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193774 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193776 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193778 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193792 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193795 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193798 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193801 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193803 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193805 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193808 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193811 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 20:02:04.196278 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193813 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193815 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193819 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193823 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193826 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193829 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.193834 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193957 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193967 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193971 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193973 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193976 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193980 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193983 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.193986 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 20:02:04.196754 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194007 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194012 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194014 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194017 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194020 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194022 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194025 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194027 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194030 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194032 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194035 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194037 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194040 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194042 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194045 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194047 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194050 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194052 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194054 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194057 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 20:02:04.197144 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194059 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194062 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194064 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194067 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194069 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194072 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194081 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194084 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194086 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194089 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194091 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194093 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194096 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194099 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194102 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194104 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194107 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194109 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194112 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194114 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 20:02:04.197626 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194117 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194119 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194122 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194125 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194127 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194130 2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194132 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194135 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194137 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194140 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194142 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194144 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194147 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194149 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194152 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194154 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194157 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194159 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194162 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 20:02:04.198127 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194169 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194172 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194174 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194177 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194179 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194182 2577 feature_gate.go:328] unrecognized feature gate: Example Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194185 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194188 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194192 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194195 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194198 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194201 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194203 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194206 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194209 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194211 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194214 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194217 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 20:02:04.198615 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:04.194220 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 20:02:04.199070 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.194225 2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 20:02:04.199070 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.195074 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 20:02:04.199070 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.197869 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 20:02:04.199431 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.199418 2577 server.go:1019] "Starting client certificate rotation" Jun 02 20:02:04.199535 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.199518 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 20:02:04.199567 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.199559 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 20:02:04.228308 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.228287 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 20:02:04.231680 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.230345 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 20:02:04.249584 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.249564 2577 log.go:25] "Validated CRI v1 runtime API" Jun 02 20:02:04.256310 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.256295 2577 log.go:25] "Validated CRI v1 image API" Jun 02 20:02:04.257535 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.257516 2577 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 20:02:04.260207 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.260175 2577 fs.go:135] Filesystem UUIDs: map[396af5f6-b669-4c79-b08f-5daa90d2fe26:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a79a7311-ac0f-43fb-8b54-4553548fdefc:/dev/nvme0n1p3] Jun 02 20:02:04.260280 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.260206 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 20:02:04.263270 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.263251 2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 20:02:04.264533 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.264425 2577 manager.go:217] Machine: {Timestamp:2026-06-02 20:02:04.263912782 +0000 UTC m=+0.461475468 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100151 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25d9738d83e8c1236ba4b6d74a998f SystemUUID:ec25d973-8d83-e8c1-236b-a4b6d74a998f BootID:b9c44272-dfed-42a0-98e5-b829f14fa523 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f1:ad:06:e8:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f1:ad:06:e8:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:25:8f:fb:7a:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 20:02:04.266161 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.266150 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 20:02:04.266246 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.266233 2577 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 20:02:04.268505 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.268482 2577 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 20:02:04.268649 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.268508 2577 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-118.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 20:02:04.268695 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.268661 2577 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 20:02:04.268695 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.268669 2577 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 20:02:04.268695 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.268676 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 20:02:04.269553 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.269542 2577 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 20:02:04.270944 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.270934 2577 state_mem.go:36] "Initialized new in-memory state store" Jun 02 20:02:04.271242 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.271231 2577 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 20:02:04.274365 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.274356 2577 kubelet.go:491] "Attempting to sync node with API server" Jun 02 20:02:04.274402 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.274373 2577 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 20:02:04.274402 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.274386 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 20:02:04.274402 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.274396 2577 kubelet.go:397] "Adding apiserver pod source" Jun 02 20:02:04.274499 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.274404 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 20:02:04.275577 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.275566 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 20:02:04.275634 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.275584 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 20:02:04.279412 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.279378 2577 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 20:02:04.282745 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.282725 2577 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 20:02:04.284375 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284357 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 20:02:04.284442 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284379 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 20:02:04.284442 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284385 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 20:02:04.284442 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284391 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 20:02:04.284442 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284399 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 20:02:04.284442 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284405 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 20:02:04.284442 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284413 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 20:02:04.284442 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284424 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 20:02:04.284442 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284434 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 20:02:04.284442 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284440 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 20:02:04.284691 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284449 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 20:02:04.284691 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.284454 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 20:02:04.285202 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.285191 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 20:02:04.285240 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.285205 2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 20:02:04.288302 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.288287 2577 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 20:02:04.288373 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.288324 2577 server.go:1295] "Started kubelet" Jun 02 20:02:04.288434 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.288399 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 20:02:04.288493 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.288452 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nj7m7" Jun 02 20:02:04.288538 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.288464 2577 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 20:02:04.288538 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.288532 2577 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 20:02:04.289160 ip-10-0-134-118 systemd[1]: Started Kubernetes Kubelet. Jun 02 20:02:04.289271 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.289165 2577 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 20:02:04.289271 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.289181 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 20:02:04.289386 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.289285 2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 20:02:04.289760 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.289692 2577 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 20:02:04.291470 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.291455 2577 server.go:317] "Adding debug handlers to kubelet server" Jun 02 20:02:04.295490 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.295475 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nj7m7" Jun 02 20:02:04.295684 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.295665 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 20:02:04.297178 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.297159 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 20:02:04.297537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.297518 2577 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 20:02:04.297537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.297533 2577 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 20:02:04.297673 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.297632 2577 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 20:02:04.297732 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.297693 2577 reconstruct.go:97] "Volume reconstruction finished" Jun 02 20:02:04.297732 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.297700 2577 reconciler.go:26] "Reconciler: start to sync state" Jun 02 20:02:04.298801 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.298778 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:04.298899 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.298892 2577 factory.go:55] Registering systemd factory Jun 02 20:02:04.298899 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.298876 2577 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 20:02:04.299024 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.298903 2577 factory.go:223] Registration of the systemd container factory successfully Jun 02 20:02:04.299177 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.299155 2577 factory.go:153] Registering CRI-O factory Jun 02 20:02:04.299177 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.299173 2577 factory.go:223] Registration of the crio container factory successfully Jun 02 20:02:04.299323 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.299222 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 20:02:04.299323 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.299238 2577 factory.go:103] Registering Raw factory Jun 02 20:02:04.299323 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.299254 2577 manager.go:1196] Started watching for new ooms in manager Jun 02 20:02:04.299574 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.297803 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-118.ec2.internal.18b55c20f3776ded default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-118.ec2.internal,UID:ip-10-0-134-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-118.ec2.internal,},FirstTimestamp:2026-06-02 20:02:04.288298477 +0000 UTC m=+0.485861162,LastTimestamp:2026-06-02 20:02:04.288298477 +0000 UTC m=+0.485861162,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-118.ec2.internal,}" Jun 02 20:02:04.300430 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.300416 2577 manager.go:319] Starting recovery of all containers Jun 02 20:02:04.309730 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.309561 2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 20:02:04.310704 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.309841 2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-118.ec2.internal\" not found" node="ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.312670 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.312647 2577 manager.go:324] Recovery completed Jun 02 20:02:04.316642 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.316628 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 20:02:04.318591 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.318575 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jun 02 20:02:04.318675 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.318606 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 20:02:04.318675 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.318621 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jun 02 20:02:04.319170 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.319148 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 20:02:04.319212 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.319170 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 20:02:04.319212 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.319195 2577 state_mem.go:36] "Initialized new in-memory state store" Jun 02 20:02:04.321651 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.321631 2577 policy_none.go:49] "None policy: Start" Jun 02 20:02:04.321651 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.321651 2577 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 20:02:04.321774 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.321668 2577 state_mem.go:35] "Initializing new in-memory state store" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.362964 2577 manager.go:341] "Starting Device Plugin manager" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.363026 2577 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.363040 2577 server.go:85] "Starting device plugin registration server" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.363288 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.363298 2577 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.363391 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.363472 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.363482 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.364011 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 20:02:04.383799 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.364069 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:04.434531 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.434466 2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 20:02:04.435709 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.435691 2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 20:02:04.435776 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.435718 2577 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 20:02:04.435776 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.435741 2577 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 20:02:04.435776 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.435750 2577 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 20:02:04.435904 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.435793 2577 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 20:02:04.438599 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.438583 2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 20:02:04.463811 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.463793 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 20:02:04.464814 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.464797 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jun 02 20:02:04.464908 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.464829 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 20:02:04.464908 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.464844 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jun 02 20:02:04.464908 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.464872 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.473403 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.473384 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.473486 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.473408 2577 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-118.ec2.internal\": node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:04.494471 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.494450 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:04.536465 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.536443 2577 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal"] Jun 02 20:02:04.536531 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.536505 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 20:02:04.537298 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.537285 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jun 02 20:02:04.537366 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.537311 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 20:02:04.537366 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.537320 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jun 02 20:02:04.538472 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.538459 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 20:02:04.538640 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.538624 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.538701 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.538661 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 20:02:04.539168 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.539145 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jun 02 20:02:04.539258 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.539180 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 20:02:04.539258 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.539186 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jun 02 20:02:04.539258 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.539207 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 20:02:04.539258 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.539191 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jun 02 20:02:04.539258 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.539216 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jun 02 20:02:04.540333 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.540312 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.540386 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.540348 2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 20:02:04.540956 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.540942 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jun 02 20:02:04.541034 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.540967 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 20:02:04.541034 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.541010 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jun 02 20:02:04.567545 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.567527 2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-118.ec2.internal\" not found" node="ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.571975 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.571961 2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-118.ec2.internal\" not found" node="ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.594918 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.594903 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:04.599193 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.599178 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f418a6cbbba733792baed213ce1222cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f418a6cbbba733792baed213ce1222cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.599240 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.599211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f418a6cbbba733792baed213ce1222cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f418a6cbbba733792baed213ce1222cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.599240 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.599231 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/029bda4e5c20ee6cc79d398e78cb1fc5-config\") pod \"kube-apiserver-proxy-ip-10-0-134-118.ec2.internal\" (UID: \"029bda4e5c20ee6cc79d398e78cb1fc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.695624 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.695544 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:04.699896 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.699881 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f418a6cbbba733792baed213ce1222cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f418a6cbbba733792baed213ce1222cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.699951 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.699908 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f418a6cbbba733792baed213ce1222cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f418a6cbbba733792baed213ce1222cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.699951 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.699926 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/029bda4e5c20ee6cc79d398e78cb1fc5-config\") pod \"kube-apiserver-proxy-ip-10-0-134-118.ec2.internal\" (UID: \"029bda4e5c20ee6cc79d398e78cb1fc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.700032 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.699953 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/029bda4e5c20ee6cc79d398e78cb1fc5-config\") pod \"kube-apiserver-proxy-ip-10-0-134-118.ec2.internal\" (UID: \"029bda4e5c20ee6cc79d398e78cb1fc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.700032 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.699979 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f418a6cbbba733792baed213ce1222cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f418a6cbbba733792baed213ce1222cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.700032 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.699979 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f418a6cbbba733792baed213ce1222cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f418a6cbbba733792baed213ce1222cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.796538 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.796496 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:04.869976 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.869943 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.876064 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:04.876040 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jun 02 20:02:04.897190 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.897162 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:04.997730 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:04.997628 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:05.098070 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.098036 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:05.198513 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.198481 2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jun 02 20:02:05.198513 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.198492 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 20:02:05.199150 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.198648 2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 20:02:05.199150 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.198655 2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 20:02:05.259818 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.259749 2577 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 20:02:05.274569 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.274532 2577 apiserver.go:52] "Watching apiserver" Jun 02 20:02:05.286503 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.286478 2577 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 20:02:05.286857 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.286831 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-sdkqb","openshift-multus/network-metrics-daemon-q8xf6","openshift-network-operator/iptables-alerter-lf27t","openshift-ovn-kubernetes/ovnkube-node-jmkdm","kube-system/konnectivity-agent-vxdnd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59","openshift-cluster-node-tuning-operator/tuned-sl6q8","openshift-dns/node-resolver-6scmv","openshift-multus/multus-5gfl5","openshift-multus/multus-additional-cni-plugins-vthxs","openshift-network-diagnostics/network-check-target-r857s"] Jun 02 20:02:05.289309 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.289285 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.290773 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.290755 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:05.290869 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.290842 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:05.291346 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.291327 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 20:02:05.291418 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.291328 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 20:02:05.291566 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.291548 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 20:02:05.291610 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.291554 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zvp99\"" Jun 02 20:02:05.292004 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.291976 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.293244 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.293227 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.294102 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.294079 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 20:02:05.294102 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.294095 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kvrlp\"" Jun 02 20:02:05.294102 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.294089 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 20:02:05.294320 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.294125 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 20:02:05.294403 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.294390 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:05.295382 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.295364 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 20:02:05.295640 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.295624 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.295717 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.295698 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 20:02:05.295776 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.295741 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gpc8z\"" Jun 02 20:02:05.295876 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.295853 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 20:02:05.295974 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.295897 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 20:02:05.295974 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.295917 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 20:02:05.295974 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.295932 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 20:02:05.296131 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.295981 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 20:02:05.296396 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.296380 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 20:02:05.296466 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.296407 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6rq4d\"" Jun 02 20:02:05.296466 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.296422 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 20:02:05.296839 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.296820 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.297401 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.297387 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 20:02:05.297471 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.297452 2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jun 02 20:02:05.297725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.297704 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 19:57:04 +0000 UTC" deadline="2028-01-04 08:51:19.044455401 +0000 UTC" Jun 02 20:02:05.297781 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.297727 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13932h49m13.746731792s" Jun 02 20:02:05.297833 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.297811 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 20:02:05.298131 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.298115 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6vvck\"" Jun 02 20:02:05.298263 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.298248 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.298830 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.298775 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 20:02:05.298910 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.298780 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 20:02:05.298910 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.298881 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nwnj8\"" Jun 02 20:02:05.298910 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.298888 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 20:02:05.299527 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.299509 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.300576 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.300553 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mnh5x\"" Jun 02 20:02:05.300651 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.300565 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 20:02:05.300651 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.300630 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.301091 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.301730 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.301761 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.302085 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.302100 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r6d6l\"" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.302324 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.302852 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-run-netns\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.302898 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-cni-netd\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.302934 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-sysctl-conf\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.302965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-run\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303021 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-run-openvswitch\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-ovnkube-config\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b8bcd73-e125-45ea-b5a5-96a894930117-tmp\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303106 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-host\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f58a7052-611f-4d3e-83b2-3f428393d528-iptables-alerter-script\") pod \"iptables-alerter-lf27t\" (UID: \"f58a7052-611f-4d3e-83b2-3f428393d528\") " pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303174 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2jvd\" (UniqueName: \"kubernetes.io/projected/f58a7052-611f-4d3e-83b2-3f428393d528-kube-api-access-q2jvd\") pod \"iptables-alerter-lf27t\" (UID: \"f58a7052-611f-4d3e-83b2-3f428393d528\") " pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303206 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-slash\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303231 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-run-systemd\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-modprobe-d\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303294 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-sys-fs\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303326 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s45t\" (UniqueName: \"kubernetes.io/projected/de05dc1e-c5cc-4145-8401-618bdf0adf45-kube-api-access-5s45t\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303360 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-sysctl-d\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-var-lib-openvswitch\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303446 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-log-socket\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303473 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-cni-bin\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303501 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-ovn-node-metrics-cert\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.303611 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-socket-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303562 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-tuned\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/221c0697-6afa-4a37-902f-3048a712c075-hosts-file\") pod \"node-resolver-6scmv\" (UID: \"221c0697-6afa-4a37-902f-3048a712c075\") " pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-run-ovn\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303676 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-run-ovn-kubernetes\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303708 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303756 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303862 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sqkhk\"" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303875 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-device-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303908 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcp2r\" (UniqueName: \"kubernetes.io/projected/9b8bcd73-e125-45ea-b5a5-96a894930117-kube-api-access-qcp2r\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5089440b-e3f8-4581-8d0d-03c318033288-serviceca\") pod \"node-ca-sdkqb\" (UID: \"5089440b-e3f8-4581-8d0d-03c318033288\") " pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.303969 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-systemd-units\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304092 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-registration-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304138 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdwjg\" (UniqueName: \"kubernetes.io/projected/221c0697-6afa-4a37-902f-3048a712c075-kube-api-access-zdwjg\") pod \"node-resolver-6scmv\" (UID: \"221c0697-6afa-4a37-902f-3048a712c075\") " pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.304378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304180 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-etc-openvswitch\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304509 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304220 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304698 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-ovnkube-script-lib\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304742 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12056731-756a-476b-ae32-1bd1e4c3420b-agent-certs\") pod \"konnectivity-agent-vxdnd\" (UID: \"12056731-756a-476b-ae32-1bd1e4c3420b\") " pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304763 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-var-lib-kubelet\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304789 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhnlc\" (UniqueName: \"kubernetes.io/projected/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-kube-api-access-xhnlc\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304833 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-etc-selinux\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304864 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-systemd\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304892 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304898 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx89b\" (UniqueName: \"kubernetes.io/projected/d34cf515-d3ea-4298-97e9-3b98e1963700-kube-api-access-hx89b\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.304930 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-kubelet\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.305029 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.304969 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:05.305598 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12056731-756a-476b-ae32-1bd1e4c3420b-konnectivity-ca\") pod \"konnectivity-agent-vxdnd\" (UID: \"12056731-756a-476b-ae32-1bd1e4c3420b\") " pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:05.305598 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305567 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-sys\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.305702 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4cvl\" (UniqueName: \"kubernetes.io/projected/5089440b-e3f8-4581-8d0d-03c318033288-kube-api-access-t4cvl\") pod \"node-ca-sdkqb\" (UID: \"5089440b-e3f8-4581-8d0d-03c318033288\") " pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.305702 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305639 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-env-overrides\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.305702 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305675 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-kubernetes\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.305821 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305699 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-lib-modules\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.305821 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305721 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f58a7052-611f-4d3e-83b2-3f428393d528-host-slash\") pod \"iptables-alerter-lf27t\" (UID: \"f58a7052-611f-4d3e-83b2-3f428393d528\") " pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.305821 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305742 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-node-log\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.305821 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-sysconfig\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.305821 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305810 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5089440b-e3f8-4581-8d0d-03c318033288-host\") pod \"node-ca-sdkqb\" (UID: \"5089440b-e3f8-4581-8d0d-03c318033288\") " pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.305976 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.305847 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/221c0697-6afa-4a37-902f-3048a712c075-tmp-dir\") pod \"node-resolver-6scmv\" (UID: \"221c0697-6afa-4a37-902f-3048a712c075\") " pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.307638 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.307618 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal"] Jun 02 20:02:05.308287 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.308271 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 20:02:05.308368 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.308355 2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jun 02 20:02:05.309786 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.309770 2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 20:02:05.318431 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.318413 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal"] Jun 02 20:02:05.318724 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.318706 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 20:02:05.332855 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.332834 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kd5fm" Jun 02 20:02:05.337879 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.337858 2577 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 20:02:05.341913 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.341896 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kd5fm" Jun 02 20:02:05.358645 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.358630 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 20:02:05.398798 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.398781 2577 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 20:02:05.406192 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406174 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-run-openvswitch\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.406289 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406199 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-ovnkube-config\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.406289 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406216 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b8bcd73-e125-45ea-b5a5-96a894930117-tmp\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.406289 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406234 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-hostroot\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.406289 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-os-release\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.406488 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406314 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-run-openvswitch\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.406488 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406356 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-host\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.406488 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406379 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f58a7052-611f-4d3e-83b2-3f428393d528-iptables-alerter-script\") pod \"iptables-alerter-lf27t\" (UID: \"f58a7052-611f-4d3e-83b2-3f428393d528\") " pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.406488 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406395 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q2jvd\" (UniqueName: \"kubernetes.io/projected/f58a7052-611f-4d3e-83b2-3f428393d528-kube-api-access-q2jvd\") pod \"iptables-alerter-lf27t\" (UID: \"f58a7052-611f-4d3e-83b2-3f428393d528\") " pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.406488 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406415 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-slash\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.406488 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406440 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-run-systemd\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.406488 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406463 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-modprobe-d\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.406488 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406488 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-run-netns\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406521 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-run-systemd\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406534 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-sys-fs\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406502 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-slash\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406562 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5s45t\" (UniqueName: \"kubernetes.io/projected/de05dc1e-c5cc-4145-8401-618bdf0adf45-kube-api-access-5s45t\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406568 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406587 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-var-lib-cni-multus\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406584 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-sys-fs\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406485 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-host\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406613 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-run-multus-certs\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406622 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-modprobe-d\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406639 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-cnibin\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406663 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4bd9970a-116d-4749-8219-ed8d4f5be7f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406685 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406710 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b79px\" (UniqueName: \"kubernetes.io/projected/79ca1d6a-f410-48c0-8217-adef4c71ea5d-kube-api-access-b79px\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406751 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406775 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-sysctl-d\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.406872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406793 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-daemon-config\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406797 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-ovnkube-config\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406823 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4bd9970a-116d-4749-8219-ed8d4f5be7f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.406832 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406847 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f58a7052-611f-4d3e-83b2-3f428393d528-iptables-alerter-script\") pod \"iptables-alerter-lf27t\" (UID: \"f58a7052-611f-4d3e-83b2-3f428393d528\") " pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406854 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-var-lib-openvswitch\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406880 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-log-socket\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.406901 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs podName:d34cf515-d3ea-4298-97e9-3b98e1963700 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:05.906878547 +0000 UTC m=+2.104441219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs") pod "network-metrics-daemon-q8xf6" (UID: "d34cf515-d3ea-4298-97e9-3b98e1963700") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406917 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-var-lib-openvswitch\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406919 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-sysctl-d\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406937 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-log-socket\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.406959 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-cni-bin\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407005 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-ovn-node-metrics-cert\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407012 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-cni-bin\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407025 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-socket-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407040 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-tuned\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407059 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-var-lib-cni-bin\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.407694 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407074 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/221c0697-6afa-4a37-902f-3048a712c075-hosts-file\") pod \"node-resolver-6scmv\" (UID: \"221c0697-6afa-4a37-902f-3048a712c075\") " pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407096 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-run-ovn\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407111 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-run-ovn-kubernetes\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407127 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407142 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-device-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407126 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/221c0697-6afa-4a37-902f-3048a712c075-hosts-file\") pod \"node-resolver-6scmv\" (UID: \"221c0697-6afa-4a37-902f-3048a712c075\") " pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407156 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-run-ovn\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407164 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qcp2r\" (UniqueName: \"kubernetes.io/projected/9b8bcd73-e125-45ea-b5a5-96a894930117-kube-api-access-qcp2r\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407196 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407190 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-device-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-conf-dir\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407220 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-run-ovn-kubernetes\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407262 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4bd9970a-116d-4749-8219-ed8d4f5be7f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407278 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-socket-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407291 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5089440b-e3f8-4581-8d0d-03c318033288-serviceca\") pod \"node-ca-sdkqb\" (UID: \"5089440b-e3f8-4581-8d0d-03c318033288\") " pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407333 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-systemd-units\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407361 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-registration-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.409008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407387 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-cnibin\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407392 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-systemd-units\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-socket-dir-parent\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407420 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-registration-dir\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407448 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zdwjg\" (UniqueName: \"kubernetes.io/projected/221c0697-6afa-4a37-902f-3048a712c075-kube-api-access-zdwjg\") pod \"node-resolver-6scmv\" (UID: \"221c0697-6afa-4a37-902f-3048a712c075\") " pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407493 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-etc-openvswitch\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407525 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407551 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-ovnkube-script-lib\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407557 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-etc-openvswitch\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407576 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12056731-756a-476b-ae32-1bd1e4c3420b-agent-certs\") pod \"konnectivity-agent-vxdnd\" (UID: \"12056731-756a-476b-ae32-1bd1e4c3420b\") " pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407601 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407604 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-etc-kubernetes\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407642 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-system-cni-dir\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407657 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5089440b-e3f8-4581-8d0d-03c318033288-serviceca\") pod \"node-ca-sdkqb\" (UID: \"5089440b-e3f8-4581-8d0d-03c318033288\") " pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407669 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-var-lib-kubelet\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407715 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xhnlc\" (UniqueName: \"kubernetes.io/projected/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-kube-api-access-xhnlc\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407738 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-etc-selinux\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.409802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407761 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-systemd\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-system-cni-dir\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407809 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-run-k8s-cni-cncf-io\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407834 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxx7v\" (UniqueName: \"kubernetes.io/projected/4bd9970a-116d-4749-8219-ed8d4f5be7f2-kube-api-access-kxx7v\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407863 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hx89b\" (UniqueName: \"kubernetes.io/projected/d34cf515-d3ea-4298-97e9-3b98e1963700-kube-api-access-hx89b\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407888 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-kubelet\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407913 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12056731-756a-476b-ae32-1bd1e4c3420b-konnectivity-ca\") pod \"konnectivity-agent-vxdnd\" (UID: \"12056731-756a-476b-ae32-1bd1e4c3420b\") " pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407940 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-sys\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-cni-dir\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.407986 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-var-lib-kubelet\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408014 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t4cvl\" (UniqueName: \"kubernetes.io/projected/5089440b-e3f8-4581-8d0d-03c318033288-kube-api-access-t4cvl\") pod \"node-ca-sdkqb\" (UID: \"5089440b-e3f8-4581-8d0d-03c318033288\") " pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408070 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-env-overrides\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408091 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-sys\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408096 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-kubernetes\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408101 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-kubelet\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408121 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-lib-modules\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408119 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-ovnkube-script-lib\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.410345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-os-release\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408200 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f58a7052-611f-4d3e-83b2-3f428393d528-host-slash\") pod \"iptables-alerter-lf27t\" (UID: \"f58a7052-611f-4d3e-83b2-3f428393d528\") " pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408211 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/de05dc1e-c5cc-4145-8401-618bdf0adf45-etc-selinux\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408224 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-node-log\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408242 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-lib-modules\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408247 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-sysconfig\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408283 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408289 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-sysconfig\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408282 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f58a7052-611f-4d3e-83b2-3f428393d528-host-slash\") pod \"iptables-alerter-lf27t\" (UID: \"f58a7052-611f-4d3e-83b2-3f428393d528\") " pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408313 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5089440b-e3f8-4581-8d0d-03c318033288-host\") pod \"node-ca-sdkqb\" (UID: \"5089440b-e3f8-4581-8d0d-03c318033288\") " pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408336 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-systemd\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408339 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ca1d6a-f410-48c0-8217-adef4c71ea5d-cni-binary-copy\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408344 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-node-log\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408380 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-kubernetes\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408379 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-var-lib-kubelet\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408390 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5089440b-e3f8-4581-8d0d-03c318033288-host\") pod \"node-ca-sdkqb\" (UID: \"5089440b-e3f8-4581-8d0d-03c318033288\") " pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408416 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/221c0697-6afa-4a37-902f-3048a712c075-tmp-dir\") pod \"node-resolver-6scmv\" (UID: \"221c0697-6afa-4a37-902f-3048a712c075\") " pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408453 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-run-netns\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.410795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408495 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-run-netns\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408539 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-cni-netd\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408577 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-sysctl-conf\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408600 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-run\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408613 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-env-overrides\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408674 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-host-cni-netd\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408716 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-run\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408745 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/221c0697-6afa-4a37-902f-3048a712c075-tmp-dir\") pod \"node-resolver-6scmv\" (UID: \"221c0697-6afa-4a37-902f-3048a712c075\") " pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408804 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-sysctl-conf\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.408836 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12056731-756a-476b-ae32-1bd1e4c3420b-konnectivity-ca\") pod \"konnectivity-agent-vxdnd\" (UID: \"12056731-756a-476b-ae32-1bd1e4c3420b\") " pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.409765 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b8bcd73-e125-45ea-b5a5-96a894930117-tmp\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.409861 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b8bcd73-e125-45ea-b5a5-96a894930117-etc-tuned\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.410072 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-ovn-node-metrics-cert\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.411279 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.410237 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12056731-756a-476b-ae32-1bd1e4c3420b-agent-certs\") pod \"konnectivity-agent-vxdnd\" (UID: \"12056731-756a-476b-ae32-1bd1e4c3420b\") " pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:05.418677 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.418656 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s45t\" (UniqueName: \"kubernetes.io/projected/de05dc1e-c5cc-4145-8401-618bdf0adf45-kube-api-access-5s45t\") pod \"aws-ebs-csi-driver-node-6nj59\" (UID: \"de05dc1e-c5cc-4145-8401-618bdf0adf45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.418761 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.418714 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdwjg\" (UniqueName: \"kubernetes.io/projected/221c0697-6afa-4a37-902f-3048a712c075-kube-api-access-zdwjg\") pod \"node-resolver-6scmv\" (UID: \"221c0697-6afa-4a37-902f-3048a712c075\") " pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.418867 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.418850 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcp2r\" (UniqueName: \"kubernetes.io/projected/9b8bcd73-e125-45ea-b5a5-96a894930117-kube-api-access-qcp2r\") pod \"tuned-sl6q8\" (UID: \"9b8bcd73-e125-45ea-b5a5-96a894930117\") " pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.419974 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.419954 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2jvd\" (UniqueName: \"kubernetes.io/projected/f58a7052-611f-4d3e-83b2-3f428393d528-kube-api-access-q2jvd\") pod \"iptables-alerter-lf27t\" (UID: \"f58a7052-611f-4d3e-83b2-3f428393d528\") " pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.422194 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.422172 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4cvl\" (UniqueName: \"kubernetes.io/projected/5089440b-e3f8-4581-8d0d-03c318033288-kube-api-access-t4cvl\") pod \"node-ca-sdkqb\" (UID: \"5089440b-e3f8-4581-8d0d-03c318033288\") " pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.422499 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.422481 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx89b\" (UniqueName: \"kubernetes.io/projected/d34cf515-d3ea-4298-97e9-3b98e1963700-kube-api-access-hx89b\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:05.422867 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.422852 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhnlc\" (UniqueName: \"kubernetes.io/projected/5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd-kube-api-access-xhnlc\") pod \"ovnkube-node-jmkdm\" (UID: \"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.438551 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.438509 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" event={"ID":"029bda4e5c20ee6cc79d398e78cb1fc5","Type":"ContainerStarted","Data":"144b57d75851383c604802ea1e5a86d07f3e5dcb20c34fce431744a6a461ac17"} Jun 02 20:02:05.439319 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.439301 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" event={"ID":"f418a6cbbba733792baed213ce1222cc","Type":"ContainerStarted","Data":"5a6687368ab2440b9926d8706b43518fd845bb64761867f62543da7bf286767b"} Jun 02 20:02:05.509859 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509812 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-hostroot\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.509859 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509838 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-os-release\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.509859 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509855 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-run-netns\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509878 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-var-lib-cni-multus\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509901 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-run-multus-certs\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509909 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-hostroot\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509925 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-cnibin\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509930 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-run-netns\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509953 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4bd9970a-116d-4749-8219-ed8d4f5be7f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509939 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-os-release\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509961 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-run-multus-certs\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509970 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-cnibin\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509978 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.509939 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-var-lib-cni-multus\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510021 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b79px\" (UniqueName: \"kubernetes.io/projected/79ca1d6a-f410-48c0-8217-adef4c71ea5d-kube-api-access-b79px\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510068 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-daemon-config\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510097 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4bd9970a-116d-4749-8219-ed8d4f5be7f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510125 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-var-lib-cni-bin\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510152 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-conf-dir\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510176 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4bd9970a-116d-4749-8219-ed8d4f5be7f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510205 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-cnibin\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510212 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-conf-dir\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510227 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-socket-dir-parent\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510176 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510257 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-etc-kubernetes\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510282 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-system-cni-dir\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510287 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-var-lib-cni-bin\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510333 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4bd9970a-116d-4749-8219-ed8d4f5be7f2-system-cni-dir\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510335 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-socket-dir-parent\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510342 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-etc-kubernetes\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510391 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-system-cni-dir\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.510492 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510290 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-cnibin\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510442 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-run-k8s-cni-cncf-io\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510451 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-system-cni-dir\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510470 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxx7v\" (UniqueName: \"kubernetes.io/projected/4bd9970a-116d-4749-8219-ed8d4f5be7f2-kube-api-access-kxx7v\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510515 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-cni-dir\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510527 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-run-k8s-cni-cncf-io\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510544 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-os-release\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510577 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510587 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-cni-dir\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510589 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4bd9970a-116d-4749-8219-ed8d4f5be7f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510595 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4bd9970a-116d-4749-8219-ed8d4f5be7f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510603 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ca1d6a-f410-48c0-8217-adef4c71ea5d-cni-binary-copy\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510649 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-var-lib-kubelet\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510697 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79ca1d6a-f410-48c0-8217-adef4c71ea5d-multus-daemon-config\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510704 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4bd9970a-116d-4749-8219-ed8d4f5be7f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510721 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-os-release\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.510739 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79ca1d6a-f410-48c0-8217-adef4c71ea5d-host-var-lib-kubelet\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.511106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.511068 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ca1d6a-f410-48c0-8217-adef4c71ea5d-cni-binary-copy\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.516556 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.516538 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 20:02:05.516607 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.516560 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 20:02:05.516607 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.516572 2577 projected.go:194] Error preparing data for projected volume kube-api-access-nb9sg for pod openshift-network-diagnostics/network-check-target-r857s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:05.516665 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.516644 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg podName:7232b3fd-c984-4ead-a075-3ee2a71cba07 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:06.016625482 +0000 UTC m=+2.214188170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nb9sg" (UniqueName: "kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg") pod "network-check-target-r857s" (UID: "7232b3fd-c984-4ead-a075-3ee2a71cba07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:05.518558 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.518540 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b79px\" (UniqueName: \"kubernetes.io/projected/79ca1d6a-f410-48c0-8217-adef4c71ea5d-kube-api-access-b79px\") pod \"multus-5gfl5\" (UID: \"79ca1d6a-f410-48c0-8217-adef4c71ea5d\") " pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.518627 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.518566 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxx7v\" (UniqueName: \"kubernetes.io/projected/4bd9970a-116d-4749-8219-ed8d4f5be7f2-kube-api-access-kxx7v\") pod \"multus-additional-cni-plugins-vthxs\" (UID: \"4bd9970a-116d-4749-8219-ed8d4f5be7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.608898 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.608874 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sdkqb" Jun 02 20:02:05.615457 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:05.615431 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5089440b_e3f8_4581_8d0d_03c318033288.slice/crio-01f355519349d042d931fab10d4ab6a9961d93a19e86ebeffcb36273fabfeb25 WatchSource:0}: Error finding container 01f355519349d042d931fab10d4ab6a9961d93a19e86ebeffcb36273fabfeb25: Status 404 returned error can't find the container with id 01f355519349d042d931fab10d4ab6a9961d93a19e86ebeffcb36273fabfeb25 Jun 02 20:02:05.635152 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.635130 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lf27t" Jun 02 20:02:05.641800 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:05.641777 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58a7052_611f_4d3e_83b2_3f428393d528.slice/crio-3e2a9ae14154b2a604f3c86f9cb90a3deff8cf796f8738c35bcb1f46c9a0afe5 WatchSource:0}: Error finding container 3e2a9ae14154b2a604f3c86f9cb90a3deff8cf796f8738c35bcb1f46c9a0afe5: Status 404 returned error can't find the container with id 3e2a9ae14154b2a604f3c86f9cb90a3deff8cf796f8738c35bcb1f46c9a0afe5 Jun 02 20:02:05.644095 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.644074 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:05.650320 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:05.650301 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d27d5f6_05ea_4b7f_86de_3c39cdf42bcd.slice/crio-5c260d6b13b4dfd4e5300d29e21602228d48a6ebeaa3cde8d3d78e5892dc6da9 WatchSource:0}: Error finding container 5c260d6b13b4dfd4e5300d29e21602228d48a6ebeaa3cde8d3d78e5892dc6da9: Status 404 returned error can't find the container with id 5c260d6b13b4dfd4e5300d29e21602228d48a6ebeaa3cde8d3d78e5892dc6da9 Jun 02 20:02:05.664835 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.664814 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:05.670374 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.670356 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" Jun 02 20:02:05.670585 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:05.670561 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12056731_756a_476b_ae32_1bd1e4c3420b.slice/crio-e97d346015e971fb923e50692da0d6193b561df4fc817460192bb9cf9b4d7ef5 WatchSource:0}: Error finding container e97d346015e971fb923e50692da0d6193b561df4fc817460192bb9cf9b4d7ef5: Status 404 returned error can't find the container with id e97d346015e971fb923e50692da0d6193b561df4fc817460192bb9cf9b4d7ef5 Jun 02 20:02:05.675703 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:05.675684 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde05dc1e_c5cc_4145_8401_618bdf0adf45.slice/crio-7b372f1c86e59c19824d9f8fd642f812968bca4b0fedda4c79a0a2aa367bf2b7 WatchSource:0}: Error finding container 7b372f1c86e59c19824d9f8fd642f812968bca4b0fedda4c79a0a2aa367bf2b7: Status 404 returned error can't find the container with id 7b372f1c86e59c19824d9f8fd642f812968bca4b0fedda4c79a0a2aa367bf2b7 Jun 02 20:02:05.675769 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.675756 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" Jun 02 20:02:05.681224 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.681208 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6scmv" Jun 02 20:02:05.681409 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:05.681383 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8bcd73_e125_45ea_b5a5_96a894930117.slice/crio-f3a9af75aad86cbb072ff9c150f582918ff0f248ecacce28462a24768e6b0956 WatchSource:0}: Error finding container f3a9af75aad86cbb072ff9c150f582918ff0f248ecacce28462a24768e6b0956: Status 404 returned error can't find the container with id f3a9af75aad86cbb072ff9c150f582918ff0f248ecacce28462a24768e6b0956 Jun 02 20:02:05.686868 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:05.686850 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod221c0697_6afa_4a37_902f_3048a712c075.slice/crio-d522fa7439143ec755133248ba97e65b31cdf02d53b7b765fe2aa7a89fd3ea2b WatchSource:0}: Error finding container d522fa7439143ec755133248ba97e65b31cdf02d53b7b765fe2aa7a89fd3ea2b: Status 404 returned error can't find the container with id d522fa7439143ec755133248ba97e65b31cdf02d53b7b765fe2aa7a89fd3ea2b Jun 02 20:02:05.687215 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.687195 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5gfl5" Jun 02 20:02:05.692372 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.692348 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vthxs" Jun 02 20:02:05.692542 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:05.692523 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ca1d6a_f410_48c0_8217_adef4c71ea5d.slice/crio-fbd9c2211027af9cda61a77a97f6c73e95409ae1219f2863856bcfdeab119a8d WatchSource:0}: Error finding container fbd9c2211027af9cda61a77a97f6c73e95409ae1219f2863856bcfdeab119a8d: Status 404 returned error can't find the container with id fbd9c2211027af9cda61a77a97f6c73e95409ae1219f2863856bcfdeab119a8d Jun 02 20:02:05.698458 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:05.698436 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bd9970a_116d_4749_8219_ed8d4f5be7f2.slice/crio-d15e866188a9fe2f097bfc5b773304080c971305ca622c7b43cba6ba87712ef8 WatchSource:0}: Error finding container d15e866188a9fe2f097bfc5b773304080c971305ca622c7b43cba6ba87712ef8: Status 404 returned error can't find the container with id d15e866188a9fe2f097bfc5b773304080c971305ca622c7b43cba6ba87712ef8 Jun 02 20:02:05.913982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:05.913759 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:05.913982 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.913877 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:05.913982 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:05.913939 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs podName:d34cf515-d3ea-4298-97e9-3b98e1963700 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:06.913924529 +0000 UTC m=+3.111487202 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs") pod "network-metrics-daemon-q8xf6" (UID: "d34cf515-d3ea-4298-97e9-3b98e1963700") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:06.114336 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.114103 2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 20:02:06.115048 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.114599 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:06.115048 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:06.114742 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 20:02:06.115048 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:06.114760 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 20:02:06.115048 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:06.114773 2577 projected.go:194] Error preparing data for projected volume kube-api-access-nb9sg for pod openshift-network-diagnostics/network-check-target-r857s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:06.115048 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:06.114829 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg podName:7232b3fd-c984-4ead-a075-3ee2a71cba07 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:07.114810841 +0000 UTC m=+3.312373519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nb9sg" (UniqueName: "kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg") pod "network-check-target-r857s" (UID: "7232b3fd-c984-4ead-a075-3ee2a71cba07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:06.191396 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.191313 2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 20:02:06.344905 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.342927 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 19:57:05 +0000 UTC" deadline="2027-11-05 16:28:47.976976339 +0000 UTC" Jun 02 20:02:06.344905 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.342962 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12500h26m41.634018647s" Jun 02 20:02:06.458104 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.457615 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5gfl5" event={"ID":"79ca1d6a-f410-48c0-8217-adef4c71ea5d","Type":"ContainerStarted","Data":"fbd9c2211027af9cda61a77a97f6c73e95409ae1219f2863856bcfdeab119a8d"} Jun 02 20:02:06.465048 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.465015 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" event={"ID":"9b8bcd73-e125-45ea-b5a5-96a894930117","Type":"ContainerStarted","Data":"f3a9af75aad86cbb072ff9c150f582918ff0f248ecacce28462a24768e6b0956"} Jun 02 20:02:06.472687 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.472657 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" event={"ID":"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd","Type":"ContainerStarted","Data":"5c260d6b13b4dfd4e5300d29e21602228d48a6ebeaa3cde8d3d78e5892dc6da9"} Jun 02 20:02:06.507398 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.507360 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lf27t" event={"ID":"f58a7052-611f-4d3e-83b2-3f428393d528","Type":"ContainerStarted","Data":"3e2a9ae14154b2a604f3c86f9cb90a3deff8cf796f8738c35bcb1f46c9a0afe5"} Jun 02 20:02:06.526019 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.525968 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sdkqb" event={"ID":"5089440b-e3f8-4581-8d0d-03c318033288","Type":"ContainerStarted","Data":"01f355519349d042d931fab10d4ab6a9961d93a19e86ebeffcb36273fabfeb25"} Jun 02 20:02:06.533453 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.533425 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerStarted","Data":"d15e866188a9fe2f097bfc5b773304080c971305ca622c7b43cba6ba87712ef8"} Jun 02 20:02:06.544514 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.544487 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6scmv" event={"ID":"221c0697-6afa-4a37-902f-3048a712c075","Type":"ContainerStarted","Data":"d522fa7439143ec755133248ba97e65b31cdf02d53b7b765fe2aa7a89fd3ea2b"} Jun 02 20:02:06.563072 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.563044 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" event={"ID":"de05dc1e-c5cc-4145-8401-618bdf0adf45","Type":"ContainerStarted","Data":"7b372f1c86e59c19824d9f8fd642f812968bca4b0fedda4c79a0a2aa367bf2b7"} Jun 02 20:02:06.566189 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.566165 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxdnd" event={"ID":"12056731-756a-476b-ae32-1bd1e4c3420b","Type":"ContainerStarted","Data":"e97d346015e971fb923e50692da0d6193b561df4fc817460192bb9cf9b4d7ef5"} Jun 02 20:02:06.922536 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:06.922150 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:06.922536 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:06.922306 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:06.922536 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:06.922377 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs podName:d34cf515-d3ea-4298-97e9-3b98e1963700 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:08.922357741 +0000 UTC m=+5.119920415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs") pod "network-metrics-daemon-q8xf6" (UID: "d34cf515-d3ea-4298-97e9-3b98e1963700") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:07.125102 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:07.123601 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:07.125102 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:07.123744 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 20:02:07.125102 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:07.123761 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 20:02:07.125102 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:07.123774 2577 projected.go:194] Error preparing data for projected volume kube-api-access-nb9sg for pod openshift-network-diagnostics/network-check-target-r857s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:07.125102 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:07.123827 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg podName:7232b3fd-c984-4ead-a075-3ee2a71cba07 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:09.123808776 +0000 UTC m=+5.321371456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nb9sg" (UniqueName: "kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg") pod "network-check-target-r857s" (UID: "7232b3fd-c984-4ead-a075-3ee2a71cba07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:07.344148 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:07.344062 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 19:57:05 +0000 UTC" deadline="2028-01-24 10:38:06.953731955 +0000 UTC" Jun 02 20:02:07.344148 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:07.344102 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14414h35m59.609634116s" Jun 02 20:02:07.437045 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:07.436942 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:07.437496 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:07.437088 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:07.437579 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:07.437514 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:07.437633 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:07.437605 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:08.938603 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:08.938566 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:08.939064 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:08.938717 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:08.939064 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:08.938798 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs podName:d34cf515-d3ea-4298-97e9-3b98e1963700 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:12.938775694 +0000 UTC m=+9.136338372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs") pod "network-metrics-daemon-q8xf6" (UID: "d34cf515-d3ea-4298-97e9-3b98e1963700") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:09.071091 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.071057 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-gcxhv"] Jun 02 20:02:09.076094 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.076062 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.076308 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.076148 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:09.140012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.139957 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/248f5d15-1445-407d-9962-2d88539c781e-dbus\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.140211 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.140048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.140211 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.140116 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/248f5d15-1445-407d-9962-2d88539c781e-kubelet-config\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.140211 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.140169 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:09.140388 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.140346 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 20:02:09.140388 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.140370 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 20:02:09.140388 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.140382 2577 projected.go:194] Error preparing data for projected volume kube-api-access-nb9sg for pod openshift-network-diagnostics/network-check-target-r857s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:09.140533 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.140439 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg podName:7232b3fd-c984-4ead-a075-3ee2a71cba07 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:13.140420125 +0000 UTC m=+9.337982812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nb9sg" (UniqueName: "kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg") pod "network-check-target-r857s" (UID: "7232b3fd-c984-4ead-a075-3ee2a71cba07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:09.241302 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.241213 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/248f5d15-1445-407d-9962-2d88539c781e-kubelet-config\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.241476 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.241310 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/248f5d15-1445-407d-9962-2d88539c781e-dbus\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.241476 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.241348 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.241476 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.241464 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:09.241657 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.241527 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret podName:248f5d15-1445-407d-9962-2d88539c781e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:09.741507296 +0000 UTC m=+5.939069970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret") pod "global-pull-secret-syncer-gcxhv" (UID: "248f5d15-1445-407d-9962-2d88539c781e") : object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:09.241657 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.241575 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/248f5d15-1445-407d-9962-2d88539c781e-kubelet-config\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.241775 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.241722 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/248f5d15-1445-407d-9962-2d88539c781e-dbus\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.436381 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.436350 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:09.436610 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.436472 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:09.436610 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.436492 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:09.436610 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.436588 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:09.744843 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:09.744806 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:09.745079 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.744961 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:09.745079 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:09.745039 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret podName:248f5d15-1445-407d-9962-2d88539c781e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:10.74502028 +0000 UTC m=+6.942582953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret") pod "global-pull-secret-syncer-gcxhv" (UID: "248f5d15-1445-407d-9962-2d88539c781e") : object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:10.752816 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:10.752781 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:10.753314 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:10.752939 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:10.753314 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:10.753013 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret podName:248f5d15-1445-407d-9962-2d88539c781e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:12.752978633 +0000 UTC m=+8.950541307 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret") pod "global-pull-secret-syncer-gcxhv" (UID: "248f5d15-1445-407d-9962-2d88539c781e") : object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:11.436597 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:11.436557 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:11.436824 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:11.436572 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:11.436824 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:11.436807 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:11.436952 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:11.436672 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:11.436952 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:11.436863 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:11.436952 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:11.436939 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:12.770621 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:12.770579 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:12.771080 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:12.770747 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:12.771080 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:12.770841 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret podName:248f5d15-1445-407d-9962-2d88539c781e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:16.770821976 +0000 UTC m=+12.968384652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret") pod "global-pull-secret-syncer-gcxhv" (UID: "248f5d15-1445-407d-9962-2d88539c781e") : object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:12.973005 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:12.972953 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:12.973227 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:12.973188 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:12.973314 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:12.973261 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs podName:d34cf515-d3ea-4298-97e9-3b98e1963700 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:20.973240845 +0000 UTC m=+17.170803524 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs") pod "network-metrics-daemon-q8xf6" (UID: "d34cf515-d3ea-4298-97e9-3b98e1963700") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:13.174666 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:13.174183 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:13.174666 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:13.174387 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 20:02:13.174666 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:13.174406 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 20:02:13.174666 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:13.174419 2577 projected.go:194] Error preparing data for projected volume kube-api-access-nb9sg for pod openshift-network-diagnostics/network-check-target-r857s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:13.174666 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:13.174479 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg podName:7232b3fd-c984-4ead-a075-3ee2a71cba07 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:21.1744583 +0000 UTC m=+17.372020974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nb9sg" (UniqueName: "kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg") pod "network-check-target-r857s" (UID: "7232b3fd-c984-4ead-a075-3ee2a71cba07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:13.436759 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:13.436521 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:13.436759 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:13.436665 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:13.436983 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:13.436755 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:13.436983 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:13.436860 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:13.436983 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:13.436907 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:13.437119 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:13.436977 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:15.436910 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:15.436875 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:15.437369 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:15.436887 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:15.437369 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:15.436981 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:15.437369 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:15.437007 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:15.437369 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:15.437090 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:15.437369 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:15.437181 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:16.802257 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:16.802216 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:16.802650 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:16.802364 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:16.802650 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:16.802440 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret podName:248f5d15-1445-407d-9962-2d88539c781e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:24.802419687 +0000 UTC m=+20.999982359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret") pod "global-pull-secret-syncer-gcxhv" (UID: "248f5d15-1445-407d-9962-2d88539c781e") : object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:17.437029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:17.436979 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:17.437193 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:17.436979 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:17.437193 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:17.437107 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:17.437312 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:17.437196 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:17.437312 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:17.436982 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:17.437312 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:17.437293 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:19.435925 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:19.435895 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:19.436378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:19.435895 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:19.436378 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:19.436021 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:19.436378 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:19.436114 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:19.436378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:19.435895 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:19.436378 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:19.436227 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:21.036342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:21.036297 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:21.036795 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:21.036422 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:21.036795 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:21.036503 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs podName:d34cf515-d3ea-4298-97e9-3b98e1963700 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:37.036486285 +0000 UTC m=+33.234048975 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs") pod "network-metrics-daemon-q8xf6" (UID: "d34cf515-d3ea-4298-97e9-3b98e1963700") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:21.237816 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:21.237774 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:21.238011 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:21.237956 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 20:02:21.238011 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:21.237980 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 20:02:21.238011 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:21.238009 2577 projected.go:194] Error preparing data for projected volume kube-api-access-nb9sg for pod openshift-network-diagnostics/network-check-target-r857s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:21.238152 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:21.238076 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg podName:7232b3fd-c984-4ead-a075-3ee2a71cba07 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:37.238055853 +0000 UTC m=+33.435618535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nb9sg" (UniqueName: "kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg") pod "network-check-target-r857s" (UID: "7232b3fd-c984-4ead-a075-3ee2a71cba07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:21.436709 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:21.436674 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:21.436868 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:21.436674 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:21.436868 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:21.436781 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:21.436868 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:21.436848 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:21.436868 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:21.436674 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:21.437087 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:21.436986 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:23.435941 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:23.435904 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:23.436406 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:23.435957 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:23.436406 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:23.436033 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:23.436406 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:23.436092 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:23.436406 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:23.436094 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:23.436406 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:23.436181 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:24.864193 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:24.864151 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:24.864597 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:24.864308 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:24.864597 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:24.864370 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret podName:248f5d15-1445-407d-9962-2d88539c781e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:40.864351429 +0000 UTC m=+37.061914103 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret") pod "global-pull-secret-syncer-gcxhv" (UID: "248f5d15-1445-407d-9962-2d88539c781e") : object "kube-system"/"original-pull-secret" not registered Jun 02 20:02:25.437090 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.436795 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:25.437185 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.436809 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:25.437185 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:25.437176 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:25.437251 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.436891 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:25.437283 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:25.437251 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:25.437327 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:25.437312 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:25.601194 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.601157 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5gfl5" event={"ID":"79ca1d6a-f410-48c0-8217-adef4c71ea5d","Type":"ContainerStarted","Data":"b46fe7451594aee504b03701991efde5afebc73020cfc734de9751c8009caf66"} Jun 02 20:02:25.602411 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.602382 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" event={"ID":"9b8bcd73-e125-45ea-b5a5-96a894930117","Type":"ContainerStarted","Data":"46c3fbf9de9f1ea35651b324831c95b847b04c97436f3d2265cc56432c0c98b7"} Jun 02 20:02:25.604754 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.604730 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" event={"ID":"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd","Type":"ContainerStarted","Data":"038023c69b2099764b7af08e30cec6201a8a8dedd1fa9b7bd11e177bf3579edd"} Jun 02 20:02:25.604754 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.604756 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" event={"ID":"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd","Type":"ContainerStarted","Data":"92fcecfbdc7545d0fac6fc4d9e9038cc080cc9351980e237327c5554ff2e59c6"} Jun 02 20:02:25.604906 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.604769 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" event={"ID":"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd","Type":"ContainerStarted","Data":"2d8a1a5cb2aeb4a3be0a8e3c18534bff9f1551df3acc82ef1a489d0b33b38db3"} Jun 02 20:02:25.604906 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.604780 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" event={"ID":"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd","Type":"ContainerStarted","Data":"1e9af4bfaa2868969992714cfbe68b74a14b8fc36b1e8a7a5a18b18f9f9edd3a"} Jun 02 20:02:25.604906 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.604788 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" event={"ID":"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd","Type":"ContainerStarted","Data":"3be69523a16920aaddec91e41cc7b199d1acb29262c55a3048c8d103ced145c2"} Jun 02 20:02:25.604906 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.604798 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" event={"ID":"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd","Type":"ContainerStarted","Data":"aac73955bc7461148ab20e0b5fd0d9f8ecdaf9d3de80bcb2e9bf508d5776aed1"} Jun 02 20:02:25.605919 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.605901 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sdkqb" event={"ID":"5089440b-e3f8-4581-8d0d-03c318033288","Type":"ContainerStarted","Data":"fdb3b34c65f8d25420ddcd8a250a0ad1ba4d66fc8355687f1a9a11eb055765f9"} Jun 02 20:02:25.607121 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.607101 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" event={"ID":"029bda4e5c20ee6cc79d398e78cb1fc5","Type":"ContainerStarted","Data":"e074a35985ee08d7824898821176282f54602da5a6d25a553d6b161aaee7b2f5"} Jun 02 20:02:25.608299 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.608276 2577 generic.go:358] "Generic (PLEG): container finished" podID="f418a6cbbba733792baed213ce1222cc" containerID="0d621eb62bc08ee9a75059481a0cd0238fa157dbd61a1c55b2d554a39483375a" exitCode=0 Jun 02 20:02:25.608382 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.608341 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" event={"ID":"f418a6cbbba733792baed213ce1222cc","Type":"ContainerDied","Data":"0d621eb62bc08ee9a75059481a0cd0238fa157dbd61a1c55b2d554a39483375a"} Jun 02 20:02:25.609613 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.609591 2577 generic.go:358] "Generic (PLEG): container finished" podID="4bd9970a-116d-4749-8219-ed8d4f5be7f2" containerID="54a44283bd771b81c24292c6f35bccb9e3e670bf6342b066f0be5b82e49916f2" exitCode=0 Jun 02 20:02:25.609718 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.609651 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerDied","Data":"54a44283bd771b81c24292c6f35bccb9e3e670bf6342b066f0be5b82e49916f2"} Jun 02 20:02:25.611040 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.611013 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6scmv" event={"ID":"221c0697-6afa-4a37-902f-3048a712c075","Type":"ContainerStarted","Data":"74fb411814410953b8ee279af37504f2b1ef12c32d09477b27dfdd321369dafa"} Jun 02 20:02:25.612271 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.612218 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" event={"ID":"de05dc1e-c5cc-4145-8401-618bdf0adf45","Type":"ContainerStarted","Data":"23fdd038db2d0f998993a39ffdf3bb38e13d93354943214fd5bd870e8df64611"} Jun 02 20:02:25.613345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.613324 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxdnd" event={"ID":"12056731-756a-476b-ae32-1bd1e4c3420b","Type":"ContainerStarted","Data":"cc76ff519362b7db0c0156da1d1f659e0fe7681a62ec50f5a4234fa7ece6bf11"} Jun 02 20:02:25.618663 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.618620 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-5gfl5" podStartSLOduration=2.6464335 podStartE2EDuration="21.618590897s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:02:05.695315823 +0000 UTC m=+1.892878495" lastFinishedPulling="2026-06-02 20:02:24.667473219 +0000 UTC m=+20.865035892" observedRunningTime="2026-06-02 20:02:25.618567584 +0000 UTC m=+21.816130280" watchObservedRunningTime="2026-06-02 20:02:25.618590897 +0000 UTC m=+21.816153592" Jun 02 20:02:25.633566 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.633532 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-6scmv" podStartSLOduration=2.657436305 podStartE2EDuration="21.633522463s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:02:05.688689839 +0000 UTC m=+1.886252512" lastFinishedPulling="2026-06-02 20:02:24.664775993 +0000 UTC m=+20.862338670" observedRunningTime="2026-06-02 20:02:25.633331052 +0000 UTC m=+21.830893747" watchObservedRunningTime="2026-06-02 20:02:25.633522463 +0000 UTC m=+21.831085158" Jun 02 20:02:25.668127 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.668089 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" podStartSLOduration=20.668076663 podStartE2EDuration="20.668076663s" podCreationTimestamp="2026-06-02 20:02:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:02:25.647187957 +0000 UTC m=+21.844750869" watchObservedRunningTime="2026-06-02 20:02:25.668076663 +0000 UTC m=+21.865639357" Jun 02 20:02:25.668287 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.668161 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sdkqb" podStartSLOduration=2.7057168750000002 podStartE2EDuration="21.668149647s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:02:05.617162392 +0000 UTC m=+1.814725065" lastFinishedPulling="2026-06-02 20:02:24.57959515 +0000 UTC m=+20.777157837" observedRunningTime="2026-06-02 20:02:25.667679839 +0000 UTC m=+21.865242535" watchObservedRunningTime="2026-06-02 20:02:25.668149647 +0000 UTC m=+21.865712342" Jun 02 20:02:25.685794 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:25.685755 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vxdnd" podStartSLOduration=2.7789701190000002 podStartE2EDuration="21.685738222s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:02:05.672943935 +0000 UTC m=+1.870506612" lastFinishedPulling="2026-06-02 20:02:24.579712037 +0000 UTC m=+20.777274715" observedRunningTime="2026-06-02 20:02:25.684768163 +0000 UTC m=+21.882330857" watchObservedRunningTime="2026-06-02 20:02:25.685738222 +0000 UTC m=+21.883300918" Jun 02 20:02:26.527666 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:26.527639 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 20:02:26.616120 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:26.616047 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" event={"ID":"f418a6cbbba733792baed213ce1222cc","Type":"ContainerStarted","Data":"1466ef190fe07c3ec3465a3c502c2d880185aef7ed45e772c9095d4f2bef69fe"} Jun 02 20:02:26.617577 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:26.617555 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" event={"ID":"de05dc1e-c5cc-4145-8401-618bdf0adf45","Type":"ContainerStarted","Data":"c7405d2e465616ebd95948c7d5c792125f2427d7812d78c8f04a15520e7801e3"} Jun 02 20:02:26.618647 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:26.618619 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lf27t" event={"ID":"f58a7052-611f-4d3e-83b2-3f428393d528","Type":"ContainerStarted","Data":"caa7b53ba29d1a4076d22af27a94227ff0687e4802e595bc4273e91fbd343a7b"} Jun 02 20:02:26.634109 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:26.634070 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" podStartSLOduration=21.634059096 podStartE2EDuration="21.634059096s" podCreationTimestamp="2026-06-02 20:02:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:02:26.633780428 +0000 UTC m=+22.831343134" watchObservedRunningTime="2026-06-02 20:02:26.634059096 +0000 UTC m=+22.831621792" Jun 02 20:02:26.634275 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:26.634228 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sl6q8" podStartSLOduration=3.7375586629999997 podStartE2EDuration="22.634224888s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:02:05.682867462 +0000 UTC m=+1.880430134" lastFinishedPulling="2026-06-02 20:02:24.57953367 +0000 UTC m=+20.777096359" observedRunningTime="2026-06-02 20:02:25.757038695 +0000 UTC m=+21.954601389" watchObservedRunningTime="2026-06-02 20:02:26.634224888 +0000 UTC m=+22.831787582" Jun 02 20:02:26.649660 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:26.649623 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lf27t" podStartSLOduration=3.622109638 podStartE2EDuration="22.649612833s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:02:05.64326728 +0000 UTC m=+1.840829953" lastFinishedPulling="2026-06-02 20:02:24.670770475 +0000 UTC m=+20.868333148" observedRunningTime="2026-06-02 20:02:26.64939433 +0000 UTC m=+22.846957022" watchObservedRunningTime="2026-06-02 20:02:26.649612833 +0000 UTC m=+22.847175529" Jun 02 20:02:27.375744 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.375606 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T20:02:26.527660951Z","UUID":"eb668319-686d-4246-92fa-20bdd955b143","Handler":null,"Name":"","Endpoint":""} Jun 02 20:02:27.377493 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.377461 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 20:02:27.377493 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.377490 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 20:02:27.436834 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.436804 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:27.436983 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.436804 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:27.436983 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:27.436918 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:27.436983 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.436804 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:27.437141 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:27.437047 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:27.437189 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:27.437143 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:27.513467 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.513440 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:27.514083 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.514066 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:27.621621 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.621598 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:27.622322 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:27.622182 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vxdnd" Jun 02 20:02:28.625700 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:28.625470 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" event={"ID":"de05dc1e-c5cc-4145-8401-618bdf0adf45","Type":"ContainerStarted","Data":"6ee357a847e8373137ae853ecdcee5c24dce5c20715371bd65af169e30a29ceb"} Jun 02 20:02:28.628962 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:28.628927 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" event={"ID":"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd","Type":"ContainerStarted","Data":"da76b7589552fb3098b6614c180a0a09984bc9ec8be103854cb3e2fad8782868"} Jun 02 20:02:28.650371 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:28.650323 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6nj59" podStartSLOduration=2.749489516 podStartE2EDuration="24.650311904s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:02:05.677589881 +0000 UTC m=+1.875152556" lastFinishedPulling="2026-06-02 20:02:27.578412271 +0000 UTC m=+23.775974944" observedRunningTime="2026-06-02 20:02:28.650157972 +0000 UTC m=+24.847720688" watchObservedRunningTime="2026-06-02 20:02:28.650311904 +0000 UTC m=+24.847874598" Jun 02 20:02:29.436131 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:29.436101 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:29.436338 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:29.436229 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:29.436338 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:29.436235 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:29.436429 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:29.436353 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:29.436429 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:29.436382 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:29.436522 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:29.436442 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:31.436632 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.436473 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:31.437257 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.436485 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:31.437257 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:31.436700 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:31.437257 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.436485 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:31.437257 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:31.436780 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:31.437257 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:31.436858 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:31.637555 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.637521 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" event={"ID":"5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd","Type":"ContainerStarted","Data":"e094253c87200f1ddc0d0995b4775f073432be7d5bfb961c4d769b5a064bc274"} Jun 02 20:02:31.637824 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.637791 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:31.637824 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.637819 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:31.639202 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.639177 2577 generic.go:358] "Generic (PLEG): container finished" podID="4bd9970a-116d-4749-8219-ed8d4f5be7f2" containerID="e24ceb1bbd85b6c5d32c9b47183c73c50f6825b0040b08609ace09a0226d1bc3" exitCode=0 Jun 02 20:02:31.639280 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.639217 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerDied","Data":"e24ceb1bbd85b6c5d32c9b47183c73c50f6825b0040b08609ace09a0226d1bc3"} Jun 02 20:02:31.652785 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.652766 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:31.667071 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:31.667031 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" podStartSLOduration=8.422872208 podStartE2EDuration="27.667018291s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:02:05.651696677 +0000 UTC m=+1.849259350" lastFinishedPulling="2026-06-02 20:02:24.895842754 +0000 UTC m=+21.093405433" observedRunningTime="2026-06-02 20:02:31.666912638 +0000 UTC m=+27.864475333" watchObservedRunningTime="2026-06-02 20:02:31.667018291 +0000 UTC m=+27.864580982" Jun 02 20:02:32.643361 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:32.643331 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerStarted","Data":"ca2d8755a64034b15819c34c666d96ea12ba0d889710ea47534ece8a1ccfff48"} Jun 02 20:02:32.644014 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:32.643976 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:32.658938 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:32.658918 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:02:33.030254 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.030220 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gcxhv"] Jun 02 20:02:33.030394 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.030343 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:33.030454 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:33.030423 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:33.032760 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.032732 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q8xf6"] Jun 02 20:02:33.032889 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.032850 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:33.032978 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:33.032956 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:33.033569 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.033548 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r857s"] Jun 02 20:02:33.033661 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.033647 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:33.033767 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:33.033746 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:33.647615 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.647396 2577 generic.go:358] "Generic (PLEG): container finished" podID="4bd9970a-116d-4749-8219-ed8d4f5be7f2" containerID="ca2d8755a64034b15819c34c666d96ea12ba0d889710ea47534ece8a1ccfff48" exitCode=0 Jun 02 20:02:33.647615 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.647571 2577 generic.go:358] "Generic (PLEG): container finished" podID="4bd9970a-116d-4749-8219-ed8d4f5be7f2" containerID="6ae70a585e1c1b7ef1d9240091303ecebc30fcfbc9155f19ad940502dc932e59" exitCode=0 Jun 02 20:02:33.647615 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.647444 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerDied","Data":"ca2d8755a64034b15819c34c666d96ea12ba0d889710ea47534ece8a1ccfff48"} Jun 02 20:02:33.648049 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:33.647639 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerDied","Data":"6ae70a585e1c1b7ef1d9240091303ecebc30fcfbc9155f19ad940502dc932e59"} Jun 02 20:02:34.437370 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:34.437343 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:34.437482 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:34.437455 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:34.437540 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:34.437504 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:34.437619 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:34.437602 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:35.436154 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:35.436113 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:35.436724 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:35.436225 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:36.436432 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:36.436399 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:36.436962 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:36.436399 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:36.436962 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:36.436510 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:02:36.436962 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:36.436606 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r857s" podUID="7232b3fd-c984-4ead-a075-3ee2a71cba07" Jun 02 20:02:37.065827 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.065798 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:37.065987 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.065916 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:37.065987 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.065963 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs podName:d34cf515-d3ea-4298-97e9-3b98e1963700 nodeName:}" failed. No retries permitted until 2026-06-02 20:03:09.065949465 +0000 UTC m=+65.263512138 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs") pod "network-metrics-daemon-q8xf6" (UID: "d34cf515-d3ea-4298-97e9-3b98e1963700") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 20:02:37.266966 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.266875 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:37.267173 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.267063 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 20:02:37.267173 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.267086 2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 20:02:37.267173 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.267098 2577 projected.go:194] Error preparing data for projected volume kube-api-access-nb9sg for pod openshift-network-diagnostics/network-check-target-r857s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:37.267173 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.267159 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg podName:7232b3fd-c984-4ead-a075-3ee2a71cba07 nodeName:}" failed. No retries permitted until 2026-06-02 20:03:09.267140376 +0000 UTC m=+65.464703052 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nb9sg" (UniqueName: "kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg") pod "network-check-target-r857s" (UID: "7232b3fd-c984-4ead-a075-3ee2a71cba07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 20:02:37.436988 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.436954 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:37.437404 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.437069 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gcxhv" podUID="248f5d15-1445-407d-9962-2d88539c781e" Jun 02 20:02:37.572413 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.572335 2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeReady" Jun 02 20:02:37.572676 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.572522 2577 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 20:02:37.641372 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.641333 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zkhfn"] Jun 02 20:02:37.672674 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.672648 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cdhs6"] Jun 02 20:02:37.672852 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.672815 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.676144 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.676096 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gclhs\"" Jun 02 20:02:37.676320 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.676288 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 20:02:37.676458 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.676440 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 20:02:37.683897 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.683876 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zkhfn"] Jun 02 20:02:37.684022 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.683902 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cdhs6"] Jun 02 20:02:37.684022 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.683984 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:37.686755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.686737 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 20:02:37.686966 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.686954 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hqb8x\"" Jun 02 20:02:37.687048 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.687031 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 20:02:37.687103 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.687034 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 20:02:37.771753 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.771723 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lht4q\" (UniqueName: \"kubernetes.io/projected/e95b758c-5d94-4f5b-83cd-cbb132ef586e-kube-api-access-lht4q\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.771945 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.771766 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e95b758c-5d94-4f5b-83cd-cbb132ef586e-config-volume\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.771945 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.771782 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.771945 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.771846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95b758c-5d94-4f5b-83cd-cbb132ef586e-tmp-dir\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.872280 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.872204 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgcs6\" (UniqueName: \"kubernetes.io/projected/2f0a22d1-2b80-4b90-ba52-b598b876a693-kube-api-access-jgcs6\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:37.872520 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.872296 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lht4q\" (UniqueName: \"kubernetes.io/projected/e95b758c-5d94-4f5b-83cd-cbb132ef586e-kube-api-access-lht4q\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.872520 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.872341 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e95b758c-5d94-4f5b-83cd-cbb132ef586e-config-volume\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.872520 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.872366 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.872520 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.872396 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95b758c-5d94-4f5b-83cd-cbb132ef586e-tmp-dir\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.872520 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.872447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:37.872778 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.872567 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 20:02:37.872778 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.872638 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls podName:e95b758c-5d94-4f5b-83cd-cbb132ef586e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:38.372616933 +0000 UTC m=+34.570179606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls") pod "dns-default-zkhfn" (UID: "e95b758c-5d94-4f5b-83cd-cbb132ef586e") : secret "dns-default-metrics-tls" not found Jun 02 20:02:37.872778 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.872763 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95b758c-5d94-4f5b-83cd-cbb132ef586e-tmp-dir\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.872931 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.872910 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e95b758c-5d94-4f5b-83cd-cbb132ef586e-config-volume\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.883400 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.883373 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lht4q\" (UniqueName: \"kubernetes.io/projected/e95b758c-5d94-4f5b-83cd-cbb132ef586e-kube-api-access-lht4q\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:37.973494 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.973460 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:37.973707 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.973512 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jgcs6\" (UniqueName: \"kubernetes.io/projected/2f0a22d1-2b80-4b90-ba52-b598b876a693-kube-api-access-jgcs6\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:37.973707 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.973584 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 20:02:37.973707 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:37.973654 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert podName:2f0a22d1-2b80-4b90-ba52-b598b876a693 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:38.47363407 +0000 UTC m=+34.671196747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert") pod "ingress-canary-cdhs6" (UID: "2f0a22d1-2b80-4b90-ba52-b598b876a693") : secret "canary-serving-cert" not found Jun 02 20:02:37.985346 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:37.985323 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgcs6\" (UniqueName: \"kubernetes.io/projected/2f0a22d1-2b80-4b90-ba52-b598b876a693-kube-api-access-jgcs6\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:38.377076 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:38.377038 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:38.377279 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:38.377223 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 20:02:38.377330 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:38.377288 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls podName:e95b758c-5d94-4f5b-83cd-cbb132ef586e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:39.377272875 +0000 UTC m=+35.574835548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls") pod "dns-default-zkhfn" (UID: "e95b758c-5d94-4f5b-83cd-cbb132ef586e") : secret "dns-default-metrics-tls" not found Jun 02 20:02:38.436684 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:38.436645 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:02:38.437043 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:38.437017 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:02:38.440518 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:38.439430 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 20:02:38.440518 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:38.439453 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 20:02:38.440518 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:38.439649 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s8fv6\"" Jun 02 20:02:38.440518 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:38.439728 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 20:02:38.440518 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:38.439947 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vzgsf\"" Jun 02 20:02:38.478266 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:38.478232 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:38.478406 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:38.478380 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 20:02:38.478476 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:38.478456 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert podName:2f0a22d1-2b80-4b90-ba52-b598b876a693 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:39.478435528 +0000 UTC m=+35.675998215 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert") pod "ingress-canary-cdhs6" (UID: "2f0a22d1-2b80-4b90-ba52-b598b876a693") : secret "canary-serving-cert" not found Jun 02 20:02:39.388124 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:39.388093 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:39.388300 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:39.388231 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 20:02:39.388300 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:39.388285 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls podName:e95b758c-5d94-4f5b-83cd-cbb132ef586e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:41.388270334 +0000 UTC m=+37.585833007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls") pod "dns-default-zkhfn" (UID: "e95b758c-5d94-4f5b-83cd-cbb132ef586e") : secret "dns-default-metrics-tls" not found Jun 02 20:02:39.436245 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:39.436213 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:39.439161 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:39.439138 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 20:02:39.488734 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:39.488696 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:39.488887 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:39.488864 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 20:02:39.489019 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:39.488926 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert podName:2f0a22d1-2b80-4b90-ba52-b598b876a693 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:41.488911243 +0000 UTC m=+37.686473915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert") pod "ingress-canary-cdhs6" (UID: "2f0a22d1-2b80-4b90-ba52-b598b876a693") : secret "canary-serving-cert" not found Jun 02 20:02:40.663354 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:40.663122 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerStarted","Data":"0078f07fcb05f81e9e73b368217a43a34d1a3cee7b049aec228f490a844097d6"} Jun 02 20:02:40.900051 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:40.900018 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:40.902829 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:40.902811 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/248f5d15-1445-407d-9962-2d88539c781e-original-pull-secret\") pod \"global-pull-secret-syncer-gcxhv\" (UID: \"248f5d15-1445-407d-9962-2d88539c781e\") " pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:40.945799 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:40.945769 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gcxhv" Jun 02 20:02:41.093526 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:41.093495 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gcxhv"] Jun 02 20:02:41.097531 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:02:41.097508 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod248f5d15_1445_407d_9962_2d88539c781e.slice/crio-dc84473adf3c8448b005f69990c616b9888f2a215e36fcffe0c5a12ce29df630 WatchSource:0}: Error finding container dc84473adf3c8448b005f69990c616b9888f2a215e36fcffe0c5a12ce29df630: Status 404 returned error can't find the container with id dc84473adf3c8448b005f69990c616b9888f2a215e36fcffe0c5a12ce29df630 Jun 02 20:02:41.403681 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:41.403605 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:41.403813 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:41.403716 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 20:02:41.403813 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:41.403780 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls podName:e95b758c-5d94-4f5b-83cd-cbb132ef586e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:45.403764802 +0000 UTC m=+41.601327474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls") pod "dns-default-zkhfn" (UID: "e95b758c-5d94-4f5b-83cd-cbb132ef586e") : secret "dns-default-metrics-tls" not found Jun 02 20:02:41.504829 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:41.504791 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:41.505012 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:41.504894 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 20:02:41.505012 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:41.504947 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert podName:2f0a22d1-2b80-4b90-ba52-b598b876a693 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:45.504931924 +0000 UTC m=+41.702494603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert") pod "ingress-canary-cdhs6" (UID: "2f0a22d1-2b80-4b90-ba52-b598b876a693") : secret "canary-serving-cert" not found Jun 02 20:02:41.668306 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:41.668271 2577 generic.go:358] "Generic (PLEG): container finished" podID="4bd9970a-116d-4749-8219-ed8d4f5be7f2" containerID="0078f07fcb05f81e9e73b368217a43a34d1a3cee7b049aec228f490a844097d6" exitCode=0 Jun 02 20:02:41.668776 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:41.668347 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerDied","Data":"0078f07fcb05f81e9e73b368217a43a34d1a3cee7b049aec228f490a844097d6"} Jun 02 20:02:41.669918 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:41.669591 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gcxhv" event={"ID":"248f5d15-1445-407d-9962-2d88539c781e","Type":"ContainerStarted","Data":"dc84473adf3c8448b005f69990c616b9888f2a215e36fcffe0c5a12ce29df630"} Jun 02 20:02:42.675721 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:42.674945 2577 generic.go:358] "Generic (PLEG): container finished" podID="4bd9970a-116d-4749-8219-ed8d4f5be7f2" containerID="936ea6b13145bd86a3828e9e43371a71bdd1fcbf77237e35416dcd69d77a0af7" exitCode=0 Jun 02 20:02:42.675721 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:42.675226 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerDied","Data":"936ea6b13145bd86a3828e9e43371a71bdd1fcbf77237e35416dcd69d77a0af7"} Jun 02 20:02:43.680744 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:43.680697 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vthxs" event={"ID":"4bd9970a-116d-4749-8219-ed8d4f5be7f2","Type":"ContainerStarted","Data":"e3d91dd08fcb15eb57a638a5b16264a02bde8f9c8ee6664188e195f1e84f5ad4"} Jun 02 20:02:43.705433 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:43.705371 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vthxs" podStartSLOduration=4.931619867 podStartE2EDuration="39.70535234s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:02:05.699813302 +0000 UTC m=+1.897375975" lastFinishedPulling="2026-06-02 20:02:40.473545774 +0000 UTC m=+36.671108448" observedRunningTime="2026-06-02 20:02:43.704541772 +0000 UTC m=+39.902104468" watchObservedRunningTime="2026-06-02 20:02:43.70535234 +0000 UTC m=+39.902915037" Jun 02 20:02:45.437808 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:45.437777 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:45.438183 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:45.437939 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 20:02:45.438183 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:45.438017 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls podName:e95b758c-5d94-4f5b-83cd-cbb132ef586e nodeName:}" failed. No retries permitted until 2026-06-02 20:02:53.437986695 +0000 UTC m=+49.635549372 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls") pod "dns-default-zkhfn" (UID: "e95b758c-5d94-4f5b-83cd-cbb132ef586e") : secret "dns-default-metrics-tls" not found Jun 02 20:02:45.538742 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:45.538709 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:45.538881 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:45.538831 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 20:02:45.538923 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:45.538883 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert podName:2f0a22d1-2b80-4b90-ba52-b598b876a693 nodeName:}" failed. No retries permitted until 2026-06-02 20:02:53.538865991 +0000 UTC m=+49.736428671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert") pod "ingress-canary-cdhs6" (UID: "2f0a22d1-2b80-4b90-ba52-b598b876a693") : secret "canary-serving-cert" not found Jun 02 20:02:45.685527 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:45.685496 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gcxhv" event={"ID":"248f5d15-1445-407d-9962-2d88539c781e","Type":"ContainerStarted","Data":"2a9dd58dd0c162c14eaee257d0451e5f03e720fc6cf3542a06f817ea673c4f23"} Jun 02 20:02:45.701887 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:45.701808 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gcxhv" podStartSLOduration=32.683367371 podStartE2EDuration="36.701793946s" podCreationTimestamp="2026-06-02 20:02:09 +0000 UTC" firstStartedPulling="2026-06-02 20:02:41.099181155 +0000 UTC m=+37.296743829" lastFinishedPulling="2026-06-02 20:02:45.117607728 +0000 UTC m=+41.315170404" observedRunningTime="2026-06-02 20:02:45.701298767 +0000 UTC m=+41.898861462" watchObservedRunningTime="2026-06-02 20:02:45.701793946 +0000 UTC m=+41.899356640" Jun 02 20:02:53.494637 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:53.494591 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:02:53.495038 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:53.494783 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 20:02:53.495038 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:53.494852 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls podName:e95b758c-5d94-4f5b-83cd-cbb132ef586e nodeName:}" failed. No retries permitted until 2026-06-02 20:03:09.494836642 +0000 UTC m=+65.692399315 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls") pod "dns-default-zkhfn" (UID: "e95b758c-5d94-4f5b-83cd-cbb132ef586e") : secret "dns-default-metrics-tls" not found Jun 02 20:02:53.595222 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:02:53.595188 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:02:53.595382 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:53.595311 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 20:02:53.595382 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:02:53.595375 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert podName:2f0a22d1-2b80-4b90-ba52-b598b876a693 nodeName:}" failed. No retries permitted until 2026-06-02 20:03:09.595356542 +0000 UTC m=+65.792919215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert") pod "ingress-canary-cdhs6" (UID: "2f0a22d1-2b80-4b90-ba52-b598b876a693") : secret "canary-serving-cert" not found Jun 02 20:03:04.659066 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:04.659011 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jmkdm" Jun 02 20:03:09.097108 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.097070 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:03:09.099396 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.099378 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 20:03:09.107282 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:09.107263 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 20:03:09.107356 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:09.107327 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs podName:d34cf515-d3ea-4298-97e9-3b98e1963700 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:13.107311768 +0000 UTC m=+129.304874441 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs") pod "network-metrics-daemon-q8xf6" (UID: "d34cf515-d3ea-4298-97e9-3b98e1963700") : secret "metrics-daemon-secret" not found Jun 02 20:03:09.298921 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.298880 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:03:09.301416 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.301396 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 20:03:09.311554 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.311539 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 20:03:09.322634 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.322607 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb9sg\" (UniqueName: \"kubernetes.io/projected/7232b3fd-c984-4ead-a075-3ee2a71cba07-kube-api-access-nb9sg\") pod \"network-check-target-r857s\" (UID: \"7232b3fd-c984-4ead-a075-3ee2a71cba07\") " pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:03:09.354404 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.354348 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vzgsf\"" Jun 02 20:03:09.362342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.362318 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:03:09.486815 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.486785 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r857s"] Jun 02 20:03:09.490598 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:03:09.490571 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7232b3fd_c984_4ead_a075_3ee2a71cba07.slice/crio-f6b9e7e66947f1c6cd59bddc634f475b1a5e69a7c6149bf2ff9ea5688ccba043 WatchSource:0}: Error finding container f6b9e7e66947f1c6cd59bddc634f475b1a5e69a7c6149bf2ff9ea5688ccba043: Status 404 returned error can't find the container with id f6b9e7e66947f1c6cd59bddc634f475b1a5e69a7c6149bf2ff9ea5688ccba043 Jun 02 20:03:09.500760 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.500739 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:03:09.500849 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:09.500838 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 20:03:09.500907 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:09.500898 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls podName:e95b758c-5d94-4f5b-83cd-cbb132ef586e nodeName:}" failed. No retries permitted until 2026-06-02 20:03:41.500884064 +0000 UTC m=+97.698446737 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls") pod "dns-default-zkhfn" (UID: "e95b758c-5d94-4f5b-83cd-cbb132ef586e") : secret "dns-default-metrics-tls" not found Jun 02 20:03:09.601619 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.601600 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:03:09.601731 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:09.601719 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 20:03:09.601779 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:09.601764 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert podName:2f0a22d1-2b80-4b90-ba52-b598b876a693 nodeName:}" failed. No retries permitted until 2026-06-02 20:03:41.601752321 +0000 UTC m=+97.799314994 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert") pod "ingress-canary-cdhs6" (UID: "2f0a22d1-2b80-4b90-ba52-b598b876a693") : secret "canary-serving-cert" not found Jun 02 20:03:09.728622 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:09.728595 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r857s" event={"ID":"7232b3fd-c984-4ead-a075-3ee2a71cba07","Type":"ContainerStarted","Data":"f6b9e7e66947f1c6cd59bddc634f475b1a5e69a7c6149bf2ff9ea5688ccba043"} Jun 02 20:03:12.735568 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:12.735530 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r857s" event={"ID":"7232b3fd-c984-4ead-a075-3ee2a71cba07","Type":"ContainerStarted","Data":"5fe9942ec0b744bd0dc70cae07a0acae568d18fdbac5a84380073655f7121f8e"} Jun 02 20:03:12.735915 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:12.735653 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:03:12.755267 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:12.755215 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r857s" podStartSLOduration=65.856049954 podStartE2EDuration="1m8.75520223s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:03:09.492326817 +0000 UTC m=+65.689889489" lastFinishedPulling="2026-06-02 20:03:12.391479092 +0000 UTC m=+68.589041765" observedRunningTime="2026-06-02 20:03:12.754615402 +0000 UTC m=+68.952178097" watchObservedRunningTime="2026-06-02 20:03:12.75520223 +0000 UTC m=+68.952764924" Jun 02 20:03:41.523515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:41.523379 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:03:41.524027 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:41.523540 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 20:03:41.524027 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:41.523617 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls podName:e95b758c-5d94-4f5b-83cd-cbb132ef586e nodeName:}" failed. No retries permitted until 2026-06-02 20:04:45.523597853 +0000 UTC m=+161.721160541 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls") pod "dns-default-zkhfn" (UID: "e95b758c-5d94-4f5b-83cd-cbb132ef586e") : secret "dns-default-metrics-tls" not found Jun 02 20:03:41.624093 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:41.624053 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:03:41.624219 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:41.624187 2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 20:03:41.624263 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:03:41.624249 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert podName:2f0a22d1-2b80-4b90-ba52-b598b876a693 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:45.62423475 +0000 UTC m=+161.821797424 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert") pod "ingress-canary-cdhs6" (UID: "2f0a22d1-2b80-4b90-ba52-b598b876a693") : secret "canary-serving-cert" not found Jun 02 20:03:43.739524 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:03:43.739497 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r857s" Jun 02 20:04:13.132489 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:13.132443 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:04:13.132958 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:13.132595 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 20:04:13.132958 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:13.132671 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs podName:d34cf515-d3ea-4298-97e9-3b98e1963700 nodeName:}" failed. No retries permitted until 2026-06-02 20:06:15.132655199 +0000 UTC m=+251.330217873 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs") pod "network-metrics-daemon-q8xf6" (UID: "d34cf515-d3ea-4298-97e9-3b98e1963700") : secret "metrics-daemon-secret" not found Jun 02 20:04:15.427307 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.427270 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-f2g7w"] Jun 02 20:04:15.428944 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.428929 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.431122 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.431101 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 02 20:04:15.431122 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.431118 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 02 20:04:15.431897 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.431880 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wf6q8\"" Jun 02 20:04:15.431982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.431915 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 02 20:04:15.431982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.431887 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 02 20:04:15.435633 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.435612 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 02 20:04:15.438654 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.438635 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-f2g7w"] Jun 02 20:04:15.540899 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.540873 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn"] Jun 02 20:04:15.542655 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.542641 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:15.545148 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.545119 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 02 20:04:15.545259 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.545226 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 02 20:04:15.545259 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.545238 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 02 20:04:15.545368 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.545267 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-znnqj\"" Jun 02 20:04:15.548290 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.548271 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf92cc3-a384-405d-b806-b183462bb67a-serving-cert\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.548373 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.548302 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecf92cc3-a384-405d-b806-b183462bb67a-trusted-ca\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.548373 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.548342 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecf92cc3-a384-405d-b806-b183462bb67a-config\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.548450 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.548412 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9ql9\" (UniqueName: \"kubernetes.io/projected/ecf92cc3-a384-405d-b806-b183462bb67a-kube-api-access-q9ql9\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.553787 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.553759 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn"] Jun 02 20:04:15.632453 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.632420 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68"] Jun 02 20:04:15.634402 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.634383 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.636817 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.636795 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 02 20:04:15.636903 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.636798 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-w27tt\"" Jun 02 20:04:15.636903 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.636828 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 02 20:04:15.637096 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.637070 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 02 20:04:15.637183 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.637107 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 02 20:04:15.641437 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.641413 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr"] Jun 02 20:04:15.644541 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.644523 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr" Jun 02 20:04:15.647068 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.647039 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xvrqv\"" Jun 02 20:04:15.647674 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.647650 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68"] Jun 02 20:04:15.648873 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.648856 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecf92cc3-a384-405d-b806-b183462bb67a-trusted-ca\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.648952 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.648884 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dt6k\" (UniqueName: \"kubernetes.io/projected/ff14193a-047a-47cb-be24-5245d110b935-kube-api-access-5dt6k\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:15.648952 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.648927 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecf92cc3-a384-405d-b806-b183462bb67a-config\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.649068 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.648952 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:15.649068 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.649013 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q9ql9\" (UniqueName: \"kubernetes.io/projected/ecf92cc3-a384-405d-b806-b183462bb67a-kube-api-access-q9ql9\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.649172 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.649103 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf92cc3-a384-405d-b806-b183462bb67a-serving-cert\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.649783 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.649757 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecf92cc3-a384-405d-b806-b183462bb67a-config\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.649938 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.649919 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecf92cc3-a384-405d-b806-b183462bb67a-trusted-ca\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.651774 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.651756 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf92cc3-a384-405d-b806-b183462bb67a-serving-cert\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.656423 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.656399 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr"] Jun 02 20:04:15.670623 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.670603 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9ql9\" (UniqueName: \"kubernetes.io/projected/ecf92cc3-a384-405d-b806-b183462bb67a-kube-api-access-q9ql9\") pod \"console-operator-86d7c6cc77-f2g7w\" (UID: \"ecf92cc3-a384-405d-b806-b183462bb67a\") " pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.737613 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.737534 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:15.750451 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.750425 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5dt6k\" (UniqueName: \"kubernetes.io/projected/ff14193a-047a-47cb-be24-5245d110b935-kube-api-access-5dt6k\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:15.750541 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.750469 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:15.750541 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.750502 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkv94\" (UniqueName: \"kubernetes.io/projected/e79dcf22-1949-43cc-b49c-6883873907f1-kube-api-access-bkv94\") pod \"network-check-source-56d84b9f44-smjxr\" (UID: \"e79dcf22-1949-43cc-b49c-6883873907f1\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr" Jun 02 20:04:15.750541 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.750530 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndj2p\" (UniqueName: \"kubernetes.io/projected/0b220d09-f221-4b6f-935e-58a4d367abd3-kube-api-access-ndj2p\") pod \"kube-storage-version-migrator-operator-96ddc787c-twl68\" (UID: \"0b220d09-f221-4b6f-935e-58a4d367abd3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.750672 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.750558 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b220d09-f221-4b6f-935e-58a4d367abd3-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-twl68\" (UID: \"0b220d09-f221-4b6f-935e-58a4d367abd3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.750672 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.750583 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b220d09-f221-4b6f-935e-58a4d367abd3-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-twl68\" (UID: \"0b220d09-f221-4b6f-935e-58a4d367abd3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.750672 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:15.750595 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 20:04:15.750672 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:15.750659 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls podName:ff14193a-047a-47cb-be24-5245d110b935 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:16.250643946 +0000 UTC m=+132.448206627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-vwmnn" (UID: "ff14193a-047a-47cb-be24-5245d110b935") : secret "samples-operator-tls" not found Jun 02 20:04:15.759562 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.759529 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dt6k\" (UniqueName: \"kubernetes.io/projected/ff14193a-047a-47cb-be24-5245d110b935-kube-api-access-5dt6k\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:15.850876 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.850845 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b220d09-f221-4b6f-935e-58a4d367abd3-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-twl68\" (UID: \"0b220d09-f221-4b6f-935e-58a4d367abd3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.851049 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.850886 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b220d09-f221-4b6f-935e-58a4d367abd3-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-twl68\" (UID: \"0b220d09-f221-4b6f-935e-58a4d367abd3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.851049 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.851020 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bkv94\" (UniqueName: \"kubernetes.io/projected/e79dcf22-1949-43cc-b49c-6883873907f1-kube-api-access-bkv94\") pod \"network-check-source-56d84b9f44-smjxr\" (UID: \"e79dcf22-1949-43cc-b49c-6883873907f1\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr" Jun 02 20:04:15.851049 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.851045 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ndj2p\" (UniqueName: \"kubernetes.io/projected/0b220d09-f221-4b6f-935e-58a4d367abd3-kube-api-access-ndj2p\") pod \"kube-storage-version-migrator-operator-96ddc787c-twl68\" (UID: \"0b220d09-f221-4b6f-935e-58a4d367abd3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.851471 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.851446 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b220d09-f221-4b6f-935e-58a4d367abd3-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-twl68\" (UID: \"0b220d09-f221-4b6f-935e-58a4d367abd3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.853062 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.853041 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b220d09-f221-4b6f-935e-58a4d367abd3-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-twl68\" (UID: \"0b220d09-f221-4b6f-935e-58a4d367abd3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.854686 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.854663 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-f2g7w"] Jun 02 20:04:15.857894 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:15.857873 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf92cc3_a384_405d_b806_b183462bb67a.slice/crio-1d39f758e0988cb2ed074e91a26379a9bf2a523f98e7ec3d9f11b6f02f748f7e WatchSource:0}: Error finding container 1d39f758e0988cb2ed074e91a26379a9bf2a523f98e7ec3d9f11b6f02f748f7e: Status 404 returned error can't find the container with id 1d39f758e0988cb2ed074e91a26379a9bf2a523f98e7ec3d9f11b6f02f748f7e Jun 02 20:04:15.859782 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.859764 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkv94\" (UniqueName: \"kubernetes.io/projected/e79dcf22-1949-43cc-b49c-6883873907f1-kube-api-access-bkv94\") pod \"network-check-source-56d84b9f44-smjxr\" (UID: \"e79dcf22-1949-43cc-b49c-6883873907f1\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr" Jun 02 20:04:15.859858 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.859812 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndj2p\" (UniqueName: \"kubernetes.io/projected/0b220d09-f221-4b6f-935e-58a4d367abd3-kube-api-access-ndj2p\") pod \"kube-storage-version-migrator-operator-96ddc787c-twl68\" (UID: \"0b220d09-f221-4b6f-935e-58a4d367abd3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.944628 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.944599 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" Jun 02 20:04:15.953335 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:15.953316 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr" Jun 02 20:04:16.068891 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:16.068860 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68"] Jun 02 20:04:16.072324 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:16.072296 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b220d09_f221_4b6f_935e_58a4d367abd3.slice/crio-2586c9f75a13d8819cca53bd10a6a59baf4c765408ad42fd03c3bfe881f041f8 WatchSource:0}: Error finding container 2586c9f75a13d8819cca53bd10a6a59baf4c765408ad42fd03c3bfe881f041f8: Status 404 returned error can't find the container with id 2586c9f75a13d8819cca53bd10a6a59baf4c765408ad42fd03c3bfe881f041f8 Jun 02 20:04:16.083368 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:16.083342 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr"] Jun 02 20:04:16.086632 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:16.086606 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode79dcf22_1949_43cc_b49c_6883873907f1.slice/crio-1ef3d8a7ff627f1c5a391741eb7390a28903273cb80835f09cda4c67b3e7814a WatchSource:0}: Error finding container 1ef3d8a7ff627f1c5a391741eb7390a28903273cb80835f09cda4c67b3e7814a: Status 404 returned error can't find the container with id 1ef3d8a7ff627f1c5a391741eb7390a28903273cb80835f09cda4c67b3e7814a Jun 02 20:04:16.254240 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:16.254158 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:16.254380 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:16.254280 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 20:04:16.254380 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:16.254336 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls podName:ff14193a-047a-47cb-be24-5245d110b935 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:17.254320942 +0000 UTC m=+133.451883616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-vwmnn" (UID: "ff14193a-047a-47cb-be24-5245d110b935") : secret "samples-operator-tls" not found Jun 02 20:04:16.854771 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:16.854732 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" event={"ID":"ecf92cc3-a384-405d-b806-b183462bb67a","Type":"ContainerStarted","Data":"1d39f758e0988cb2ed074e91a26379a9bf2a523f98e7ec3d9f11b6f02f748f7e"} Jun 02 20:04:16.856196 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:16.856159 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr" event={"ID":"e79dcf22-1949-43cc-b49c-6883873907f1","Type":"ContainerStarted","Data":"d567344e68d69dd65a224b86fe1316e0025822a78858077f3fa0951496a1dfb5"} Jun 02 20:04:16.856196 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:16.856196 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr" event={"ID":"e79dcf22-1949-43cc-b49c-6883873907f1","Type":"ContainerStarted","Data":"1ef3d8a7ff627f1c5a391741eb7390a28903273cb80835f09cda4c67b3e7814a"} Jun 02 20:04:16.857273 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:16.857237 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" event={"ID":"0b220d09-f221-4b6f-935e-58a4d367abd3","Type":"ContainerStarted","Data":"2586c9f75a13d8819cca53bd10a6a59baf4c765408ad42fd03c3bfe881f041f8"} Jun 02 20:04:16.872095 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:16.872049 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-smjxr" podStartSLOduration=1.872033201 podStartE2EDuration="1.872033201s" podCreationTimestamp="2026-06-02 20:04:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:04:16.87120084 +0000 UTC m=+133.068763535" watchObservedRunningTime="2026-06-02 20:04:16.872033201 +0000 UTC m=+133.069595900" Jun 02 20:04:17.262553 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.262512 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:17.262734 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:17.262642 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 20:04:17.262734 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:17.262726 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls podName:ff14193a-047a-47cb-be24-5245d110b935 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:19.262701067 +0000 UTC m=+135.460263756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-vwmnn" (UID: "ff14193a-047a-47cb-be24-5245d110b935") : secret "samples-operator-tls" not found Jun 02 20:04:17.326438 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.326399 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cf56fffcf-glffz"] Jun 02 20:04:17.329440 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.329409 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.331852 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.331766 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 20:04:17.332069 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.332051 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 20:04:17.332155 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.332085 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gj8r6\"" Jun 02 20:04:17.332213 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.332163 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 20:04:17.338636 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.338616 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 20:04:17.340884 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.340859 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cf56fffcf-glffz"] Jun 02 20:04:17.464119 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.464067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-bound-sa-token\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.464342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.464204 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-image-registry-private-configuration\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.464342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.464278 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949edd5e-089b-4737-8e96-335fc4a3e472-ca-trust-extracted\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.464342 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.464329 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.464546 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.464361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-installation-pull-secrets\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.464546 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.464398 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgl89\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-kube-api-access-qgl89\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.464546 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.464427 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-trusted-ca\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.464664 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.464579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-registry-certificates\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.565294 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.565224 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949edd5e-089b-4737-8e96-335fc4a3e472-ca-trust-extracted\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.565294 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.565256 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.565294 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.565281 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-installation-pull-secrets\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.565652 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.565308 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qgl89\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-kube-api-access-qgl89\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.565652 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.565334 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-trusted-ca\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.565652 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.565359 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-registry-certificates\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.565652 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:17.565400 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 20:04:17.565652 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:17.565424 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf56fffcf-glffz: secret "image-registry-tls" not found Jun 02 20:04:17.565652 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:17.565482 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls podName:949edd5e-089b-4737-8e96-335fc4a3e472 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:18.065464953 +0000 UTC m=+134.263027629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls") pod "image-registry-6cf56fffcf-glffz" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472") : secret "image-registry-tls" not found Jun 02 20:04:17.565652 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.565402 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-bound-sa-token\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.566052 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.565738 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949edd5e-089b-4737-8e96-335fc4a3e472-ca-trust-extracted\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.566052 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.565806 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-image-registry-private-configuration\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.566158 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.566109 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-registry-certificates\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.566768 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.566742 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-trusted-ca\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.568471 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.568440 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-image-registry-private-configuration\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.568639 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.568614 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-installation-pull-secrets\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.576080 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.576032 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-bound-sa-token\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.576080 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.576038 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgl89\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-kube-api-access-qgl89\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:17.861321 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.861263 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" event={"ID":"ecf92cc3-a384-405d-b806-b183462bb67a","Type":"ContainerStarted","Data":"ccf71f635d551a7166b1069bbe1411d448609dc668f51f482358d65d63531f42"} Jun 02 20:04:17.878878 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:17.878825 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" podStartSLOduration=1.004018355 podStartE2EDuration="2.878810807s" podCreationTimestamp="2026-06-02 20:04:15 +0000 UTC" firstStartedPulling="2026-06-02 20:04:15.859904397 +0000 UTC m=+132.057467070" lastFinishedPulling="2026-06-02 20:04:17.734696843 +0000 UTC m=+133.932259522" observedRunningTime="2026-06-02 20:04:17.878527607 +0000 UTC m=+134.076090302" watchObservedRunningTime="2026-06-02 20:04:17.878810807 +0000 UTC m=+134.076373502" Jun 02 20:04:18.070253 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:18.070217 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:18.070417 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:18.070380 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 20:04:18.070417 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:18.070405 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf56fffcf-glffz: secret "image-registry-tls" not found Jun 02 20:04:18.070506 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:18.070469 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls podName:949edd5e-089b-4737-8e96-335fc4a3e472 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:19.070453382 +0000 UTC m=+135.268016055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls") pod "image-registry-6cf56fffcf-glffz" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472") : secret "image-registry-tls" not found Jun 02 20:04:18.864608 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:18.864580 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/0.log" Jun 02 20:04:18.865041 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:18.864621 2577 generic.go:358] "Generic (PLEG): container finished" podID="ecf92cc3-a384-405d-b806-b183462bb67a" containerID="ccf71f635d551a7166b1069bbe1411d448609dc668f51f482358d65d63531f42" exitCode=255 Jun 02 20:04:18.865041 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:18.864701 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" event={"ID":"ecf92cc3-a384-405d-b806-b183462bb67a","Type":"ContainerDied","Data":"ccf71f635d551a7166b1069bbe1411d448609dc668f51f482358d65d63531f42"} Jun 02 20:04:18.865041 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:18.864922 2577 scope.go:117] "RemoveContainer" containerID="ccf71f635d551a7166b1069bbe1411d448609dc668f51f482358d65d63531f42" Jun 02 20:04:18.866076 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:18.866056 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" event={"ID":"0b220d09-f221-4b6f-935e-58a4d367abd3","Type":"ContainerStarted","Data":"7c57bb075c5ada2a38af35f1026c4039488dc31476ebeeac666d9a63248630cf"} Jun 02 20:04:18.896380 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:18.896340 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" podStartSLOduration=1.717016078 podStartE2EDuration="3.896326909s" podCreationTimestamp="2026-06-02 20:04:15 +0000 UTC" firstStartedPulling="2026-06-02 20:04:16.074108191 +0000 UTC m=+132.271670878" lastFinishedPulling="2026-06-02 20:04:18.253419032 +0000 UTC m=+134.450981709" observedRunningTime="2026-06-02 20:04:18.895441297 +0000 UTC m=+135.093003993" watchObservedRunningTime="2026-06-02 20:04:18.896326909 +0000 UTC m=+135.093889603" Jun 02 20:04:19.078747 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:19.078712 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:19.078881 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:19.078835 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 20:04:19.078881 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:19.078849 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf56fffcf-glffz: secret "image-registry-tls" not found Jun 02 20:04:19.078954 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:19.078908 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls podName:949edd5e-089b-4737-8e96-335fc4a3e472 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:21.078889667 +0000 UTC m=+137.276452344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls") pod "image-registry-6cf56fffcf-glffz" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472") : secret "image-registry-tls" not found Jun 02 20:04:19.281371 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:19.281344 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:19.281553 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:19.281514 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 20:04:19.281623 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:19.281591 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls podName:ff14193a-047a-47cb-be24-5245d110b935 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:23.281569761 +0000 UTC m=+139.479132456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-vwmnn" (UID: "ff14193a-047a-47cb-be24-5245d110b935") : secret "samples-operator-tls" not found Jun 02 20:04:19.870267 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:19.870240 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:04:19.870638 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:19.870627 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/0.log" Jun 02 20:04:19.870689 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:19.870659 2577 generic.go:358] "Generic (PLEG): container finished" podID="ecf92cc3-a384-405d-b806-b183462bb67a" containerID="f964a8efa0b89b100aec1286fae2bfe3ca7c59548bb71347d54fdebdde0edd9a" exitCode=255 Jun 02 20:04:19.870740 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:19.870688 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" event={"ID":"ecf92cc3-a384-405d-b806-b183462bb67a","Type":"ContainerDied","Data":"f964a8efa0b89b100aec1286fae2bfe3ca7c59548bb71347d54fdebdde0edd9a"} Jun 02 20:04:19.870790 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:19.870746 2577 scope.go:117] "RemoveContainer" containerID="ccf71f635d551a7166b1069bbe1411d448609dc668f51f482358d65d63531f42" Jun 02 20:04:19.871031 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:19.871013 2577 scope.go:117] "RemoveContainer" containerID="f964a8efa0b89b100aec1286fae2bfe3ca7c59548bb71347d54fdebdde0edd9a" Jun 02 20:04:19.871234 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:19.871211 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f2g7w_openshift-console-operator(ecf92cc3-a384-405d-b806-b183462bb67a)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" podUID="ecf92cc3-a384-405d-b806-b183462bb67a" Jun 02 20:04:20.874314 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:20.874282 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:04:20.874750 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:20.874739 2577 scope.go:117] "RemoveContainer" containerID="f964a8efa0b89b100aec1286fae2bfe3ca7c59548bb71347d54fdebdde0edd9a" Jun 02 20:04:20.874967 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:20.874940 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f2g7w_openshift-console-operator(ecf92cc3-a384-405d-b806-b183462bb67a)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" podUID="ecf92cc3-a384-405d-b806-b183462bb67a" Jun 02 20:04:20.916071 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:20.916032 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c"] Jun 02 20:04:20.917913 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:20.917881 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c" Jun 02 20:04:20.920532 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:20.920511 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 02 20:04:20.920532 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:20.920528 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lqzjk\"" Jun 02 20:04:20.920721 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:20.920513 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 02 20:04:20.932017 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:20.931980 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c"] Jun 02 20:04:20.995401 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:20.995378 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5f5p\" (UniqueName: \"kubernetes.io/projected/b1ab5cf1-3968-421f-b574-74aef77fd2b5-kube-api-access-g5f5p\") pod \"migrator-96c6f749-25l5c\" (UID: \"b1ab5cf1-3968-421f-b574-74aef77fd2b5\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c" Jun 02 20:04:21.096659 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:21.096627 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:21.096659 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:21.096665 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g5f5p\" (UniqueName: \"kubernetes.io/projected/b1ab5cf1-3968-421f-b574-74aef77fd2b5-kube-api-access-g5f5p\") pod \"migrator-96c6f749-25l5c\" (UID: \"b1ab5cf1-3968-421f-b574-74aef77fd2b5\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c" Jun 02 20:04:21.096852 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:21.096777 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 20:04:21.096852 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:21.096797 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf56fffcf-glffz: secret "image-registry-tls" not found Jun 02 20:04:21.096852 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:21.096847 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls podName:949edd5e-089b-4737-8e96-335fc4a3e472 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:25.096831756 +0000 UTC m=+141.294394428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls") pod "image-registry-6cf56fffcf-glffz" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472") : secret "image-registry-tls" not found Jun 02 20:04:21.104742 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:21.104714 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5f5p\" (UniqueName: \"kubernetes.io/projected/b1ab5cf1-3968-421f-b574-74aef77fd2b5-kube-api-access-g5f5p\") pod \"migrator-96c6f749-25l5c\" (UID: \"b1ab5cf1-3968-421f-b574-74aef77fd2b5\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c" Jun 02 20:04:21.226883 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:21.226848 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c" Jun 02 20:04:21.346532 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:21.346496 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c"] Jun 02 20:04:21.349438 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:21.349410 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ab5cf1_3968_421f_b574_74aef77fd2b5.slice/crio-0d71c9fd97c15de18531103e715b2ef4ea25b7c11cd9140fc596cbc10883b551 WatchSource:0}: Error finding container 0d71c9fd97c15de18531103e715b2ef4ea25b7c11cd9140fc596cbc10883b551: Status 404 returned error can't find the container with id 0d71c9fd97c15de18531103e715b2ef4ea25b7c11cd9140fc596cbc10883b551 Jun 02 20:04:21.876894 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:21.876861 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c" event={"ID":"b1ab5cf1-3968-421f-b574-74aef77fd2b5","Type":"ContainerStarted","Data":"0d71c9fd97c15de18531103e715b2ef4ea25b7c11cd9140fc596cbc10883b551"} Jun 02 20:04:22.153594 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:22.153521 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6scmv_221c0697-6afa-4a37-902f-3048a712c075/dns-node-resolver/0.log" Jun 02 20:04:22.880438 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:22.880409 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c" event={"ID":"b1ab5cf1-3968-421f-b574-74aef77fd2b5","Type":"ContainerStarted","Data":"506e3fa555fb922f241408430b612e61e821ea7fee2e9aeb9761bfb1439da2bb"} Jun 02 20:04:23.153911 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:23.153838 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sdkqb_5089440b-e3f8-4581-8d0d-03c318033288/node-ca/0.log" Jun 02 20:04:23.313461 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:23.313421 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:23.313606 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:23.313561 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 20:04:23.313658 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:23.313623 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls podName:ff14193a-047a-47cb-be24-5245d110b935 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:31.313607665 +0000 UTC m=+147.511170338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-vwmnn" (UID: "ff14193a-047a-47cb-be24-5245d110b935") : secret "samples-operator-tls" not found Jun 02 20:04:23.884220 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:23.884187 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c" event={"ID":"b1ab5cf1-3968-421f-b574-74aef77fd2b5","Type":"ContainerStarted","Data":"4b3d29aacafe29eb471d930230f9d990882e1a53cb7b0428735f62c288583b7a"} Jun 02 20:04:23.904959 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:23.904919 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-25l5c" podStartSLOduration=2.439019337 podStartE2EDuration="3.904906117s" podCreationTimestamp="2026-06-02 20:04:20 +0000 UTC" firstStartedPulling="2026-06-02 20:04:21.351676781 +0000 UTC m=+137.549239455" lastFinishedPulling="2026-06-02 20:04:22.817563559 +0000 UTC m=+139.015126235" observedRunningTime="2026-06-02 20:04:23.903786113 +0000 UTC m=+140.101348831" watchObservedRunningTime="2026-06-02 20:04:23.904906117 +0000 UTC m=+140.102468846" Jun 02 20:04:25.127325 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:25.127282 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:25.127674 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:25.127427 2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 20:04:25.127674 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:25.127446 2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf56fffcf-glffz: secret "image-registry-tls" not found Jun 02 20:04:25.127674 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:25.127498 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls podName:949edd5e-089b-4737-8e96-335fc4a3e472 nodeName:}" failed. No retries permitted until 2026-06-02 20:04:33.127483265 +0000 UTC m=+149.325045943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls") pod "image-registry-6cf56fffcf-glffz" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472") : secret "image-registry-tls" not found Jun 02 20:04:25.738432 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:25.738398 2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:25.738736 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:25.738723 2577 scope.go:117] "RemoveContainer" containerID="f964a8efa0b89b100aec1286fae2bfe3ca7c59548bb71347d54fdebdde0edd9a" Jun 02 20:04:25.738889 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:25.738872 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f2g7w_openshift-console-operator(ecf92cc3-a384-405d-b806-b183462bb67a)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" podUID="ecf92cc3-a384-405d-b806-b183462bb67a" Jun 02 20:04:27.861894 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:27.861860 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:27.862381 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:27.862292 2577 scope.go:117] "RemoveContainer" containerID="f964a8efa0b89b100aec1286fae2bfe3ca7c59548bb71347d54fdebdde0edd9a" Jun 02 20:04:27.862509 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:27.862488 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-f2g7w_openshift-console-operator(ecf92cc3-a384-405d-b806-b183462bb67a)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" podUID="ecf92cc3-a384-405d-b806-b183462bb67a" Jun 02 20:04:31.376051 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:31.376010 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:31.378326 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:31.378307 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff14193a-047a-47cb-be24-5245d110b935-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-vwmnn\" (UID: \"ff14193a-047a-47cb-be24-5245d110b935\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:31.450531 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:31.450496 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" Jun 02 20:04:31.570017 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:31.569975 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn"] Jun 02 20:04:31.904207 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:31.904172 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" event={"ID":"ff14193a-047a-47cb-be24-5245d110b935","Type":"ContainerStarted","Data":"c3b6e4c4054b04ca67578bd85d96ef3272e32f773e132d58635ecc1ede0df5df"} Jun 02 20:04:33.190847 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.190799 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:33.193462 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.193439 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") pod \"image-registry-6cf56fffcf-glffz\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:33.242475 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.242447 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:33.487218 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.487187 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cf56fffcf-glffz"] Jun 02 20:04:33.490196 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:33.490160 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949edd5e_089b_4737_8e96_335fc4a3e472.slice/crio-190066732bd9fd421949a21ed8aeb0f19dc8c31a238d34625267951f0c5b81c9 WatchSource:0}: Error finding container 190066732bd9fd421949a21ed8aeb0f19dc8c31a238d34625267951f0c5b81c9: Status 404 returned error can't find the container with id 190066732bd9fd421949a21ed8aeb0f19dc8c31a238d34625267951f0c5b81c9 Jun 02 20:04:33.910139 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.910060 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" event={"ID":"ff14193a-047a-47cb-be24-5245d110b935","Type":"ContainerStarted","Data":"4296d99cee0da97807b556a225c4431aceb2b44f901edd86f164d651aa1e94a0"} Jun 02 20:04:33.910139 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.910099 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" event={"ID":"ff14193a-047a-47cb-be24-5245d110b935","Type":"ContainerStarted","Data":"ae9b64b5a3810a58070c6bc0244cedba5922f10efcc6ba3555e6d4d41a17cced"} Jun 02 20:04:33.911248 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.911222 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" event={"ID":"949edd5e-089b-4737-8e96-335fc4a3e472","Type":"ContainerStarted","Data":"37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04"} Jun 02 20:04:33.911248 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.911248 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" event={"ID":"949edd5e-089b-4737-8e96-335fc4a3e472","Type":"ContainerStarted","Data":"190066732bd9fd421949a21ed8aeb0f19dc8c31a238d34625267951f0c5b81c9"} Jun 02 20:04:33.911414 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.911335 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:33.931409 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.931361 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-vwmnn" podStartSLOduration=17.149859256 podStartE2EDuration="18.931349061s" podCreationTimestamp="2026-06-02 20:04:15 +0000 UTC" firstStartedPulling="2026-06-02 20:04:31.609909299 +0000 UTC m=+147.807471972" lastFinishedPulling="2026-06-02 20:04:33.391399101 +0000 UTC m=+149.588961777" observedRunningTime="2026-06-02 20:04:33.929909612 +0000 UTC m=+150.127472307" watchObservedRunningTime="2026-06-02 20:04:33.931349061 +0000 UTC m=+150.128911756" Jun 02 20:04:33.958723 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:33.958670 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" podStartSLOduration=16.958654906 podStartE2EDuration="16.958654906s" podCreationTimestamp="2026-06-02 20:04:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:04:33.958590125 +0000 UTC m=+150.156152834" watchObservedRunningTime="2026-06-02 20:04:33.958654906 +0000 UTC m=+150.156217602" Jun 02 20:04:40.437237 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:40.437207 2577 scope.go:117] "RemoveContainer" containerID="f964a8efa0b89b100aec1286fae2bfe3ca7c59548bb71347d54fdebdde0edd9a" Jun 02 20:04:40.684333 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:40.684292 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zkhfn" podUID="e95b758c-5d94-4f5b-83cd-cbb132ef586e" Jun 02 20:04:40.693471 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:40.693412 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-cdhs6" podUID="2f0a22d1-2b80-4b90-ba52-b598b876a693" Jun 02 20:04:40.933491 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:40.933467 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:04:40.933645 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:40.933523 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" event={"ID":"ecf92cc3-a384-405d-b806-b183462bb67a","Type":"ContainerStarted","Data":"3296344d494ebdff4c3138a25759ba2c063292710dae52c637161af332b3b93e"} Jun 02 20:04:40.933645 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:40.933561 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zkhfn" Jun 02 20:04:40.933953 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:40.933933 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:41.316635 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:41.316606 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-86d7c6cc77-f2g7w" Jun 02 20:04:41.458561 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:41.458521 2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-q8xf6" podUID="d34cf515-d3ea-4298-97e9-3b98e1963700" Jun 02 20:04:42.603501 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.603428 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cf56fffcf-glffz"] Jun 02 20:04:42.605108 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.605081 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nxgbz"] Jun 02 20:04:42.607771 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.607755 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.610208 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.610184 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 20:04:42.610307 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.610235 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 20:04:42.610307 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.610248 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 20:04:42.610307 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.610187 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 20:04:42.614667 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.614648 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9rtd6\"" Jun 02 20:04:42.624661 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.624642 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nxgbz"] Jun 02 20:04:42.667261 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.667235 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f95f22ec-1657-4507-af20-664bf4e3f7d3-data-volume\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.667395 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.667270 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6wgx\" (UniqueName: \"kubernetes.io/projected/f95f22ec-1657-4507-af20-664bf4e3f7d3-kube-api-access-d6wgx\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.667395 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.667321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f95f22ec-1657-4507-af20-664bf4e3f7d3-crio-socket\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.667395 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.667336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f95f22ec-1657-4507-af20-664bf4e3f7d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.667395 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.667372 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f95f22ec-1657-4507-af20-664bf4e3f7d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.768185 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.768156 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f95f22ec-1657-4507-af20-664bf4e3f7d3-crio-socket\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.768185 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.768187 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f95f22ec-1657-4507-af20-664bf4e3f7d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.768373 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.768240 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f95f22ec-1657-4507-af20-664bf4e3f7d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.768373 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.768272 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f95f22ec-1657-4507-af20-664bf4e3f7d3-crio-socket\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.768373 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.768283 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f95f22ec-1657-4507-af20-664bf4e3f7d3-data-volume\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.768373 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.768325 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d6wgx\" (UniqueName: \"kubernetes.io/projected/f95f22ec-1657-4507-af20-664bf4e3f7d3-kube-api-access-d6wgx\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.768700 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.768678 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f95f22ec-1657-4507-af20-664bf4e3f7d3-data-volume\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.768840 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.768824 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f95f22ec-1657-4507-af20-664bf4e3f7d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.770577 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.770555 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f95f22ec-1657-4507-af20-664bf4e3f7d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.776510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.776488 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6wgx\" (UniqueName: \"kubernetes.io/projected/f95f22ec-1657-4507-af20-664bf4e3f7d3-kube-api-access-d6wgx\") pod \"insights-runtime-extractor-nxgbz\" (UID: \"f95f22ec-1657-4507-af20-664bf4e3f7d3\") " pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:42.918366 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:42.918341 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nxgbz" Jun 02 20:04:43.045086 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:43.045056 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nxgbz"] Jun 02 20:04:43.049520 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:43.049493 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95f22ec_1657_4507_af20_664bf4e3f7d3.slice/crio-997e30785dc7ba3e9f32e9aed635d03689306407595df52d04e58e3fbf788ce8 WatchSource:0}: Error finding container 997e30785dc7ba3e9f32e9aed635d03689306407595df52d04e58e3fbf788ce8: Status 404 returned error can't find the container with id 997e30785dc7ba3e9f32e9aed635d03689306407595df52d04e58e3fbf788ce8 Jun 02 20:04:43.942975 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:43.942938 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxgbz" event={"ID":"f95f22ec-1657-4507-af20-664bf4e3f7d3","Type":"ContainerStarted","Data":"3fa16dbb3e747a4118eedee5b69ca6d1be9b729fe1b1c8110a7690d0b6a6fbe0"} Jun 02 20:04:43.942975 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:43.942977 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxgbz" event={"ID":"f95f22ec-1657-4507-af20-664bf4e3f7d3","Type":"ContainerStarted","Data":"623c499aa67d228b8d44f85d6c991545101f95a795749ce81b0dd3651c2b80d3"} Jun 02 20:04:43.943437 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:43.942987 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxgbz" event={"ID":"f95f22ec-1657-4507-af20-664bf4e3f7d3","Type":"ContainerStarted","Data":"997e30785dc7ba3e9f32e9aed635d03689306407595df52d04e58e3fbf788ce8"} Jun 02 20:04:45.590886 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.590855 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:04:45.593061 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.593044 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e95b758c-5d94-4f5b-83cd-cbb132ef586e-metrics-tls\") pod \"dns-default-zkhfn\" (UID: \"e95b758c-5d94-4f5b-83cd-cbb132ef586e\") " pod="openshift-dns/dns-default-zkhfn" Jun 02 20:04:45.691328 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.691305 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:04:45.693551 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.693527 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f0a22d1-2b80-4b90-ba52-b598b876a693-cert\") pod \"ingress-canary-cdhs6\" (UID: \"2f0a22d1-2b80-4b90-ba52-b598b876a693\") " pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:04:45.736959 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.736935 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gclhs\"" Jun 02 20:04:45.745056 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.745031 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zkhfn" Jun 02 20:04:45.883312 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.883280 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zkhfn"] Jun 02 20:04:45.886268 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:45.886241 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95b758c_5d94_4f5b_83cd_cbb132ef586e.slice/crio-ea044a48689b27cc5931aead3d7d895961171d9e6c881014729bd250bed174b7 WatchSource:0}: Error finding container ea044a48689b27cc5931aead3d7d895961171d9e6c881014729bd250bed174b7: Status 404 returned error can't find the container with id ea044a48689b27cc5931aead3d7d895961171d9e6c881014729bd250bed174b7 Jun 02 20:04:45.950649 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.950571 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxgbz" event={"ID":"f95f22ec-1657-4507-af20-664bf4e3f7d3","Type":"ContainerStarted","Data":"c87ffcb52a75748e74a7ebca142ccc64840d41a170e5908d257067abdc7ef4ed"} Jun 02 20:04:45.951588 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.951566 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zkhfn" event={"ID":"e95b758c-5d94-4f5b-83cd-cbb132ef586e","Type":"ContainerStarted","Data":"ea044a48689b27cc5931aead3d7d895961171d9e6c881014729bd250bed174b7"} Jun 02 20:04:45.969806 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:45.969760 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nxgbz" podStartSLOduration=1.6829117359999999 podStartE2EDuration="3.969751252s" podCreationTimestamp="2026-06-02 20:04:42 +0000 UTC" firstStartedPulling="2026-06-02 20:04:43.095902974 +0000 UTC m=+159.293465647" lastFinishedPulling="2026-06-02 20:04:45.382742491 +0000 UTC m=+161.580305163" observedRunningTime="2026-06-02 20:04:45.969326632 +0000 UTC m=+162.166889326" watchObservedRunningTime="2026-06-02 20:04:45.969751252 +0000 UTC m=+162.167313947" Jun 02 20:04:47.960124 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:47.960076 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zkhfn" event={"ID":"e95b758c-5d94-4f5b-83cd-cbb132ef586e","Type":"ContainerStarted","Data":"afc1a5e5218e6cbda5837805b8bc3f56e9c5719e4d45f28ee7d0357799bed357"} Jun 02 20:04:47.960124 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:47.960127 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zkhfn" event={"ID":"e95b758c-5d94-4f5b-83cd-cbb132ef586e","Type":"ContainerStarted","Data":"76274a868b9538ce99e2d4b62933787c8afbf64a7d5f978383751c2313764985"} Jun 02 20:04:47.960556 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:47.960224 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zkhfn" Jun 02 20:04:47.980065 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:47.980022 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-zkhfn" podStartSLOduration=129.310607886 podStartE2EDuration="2m10.979984736s" podCreationTimestamp="2026-06-02 20:02:37 +0000 UTC" firstStartedPulling="2026-06-02 20:04:45.887988147 +0000 UTC m=+162.085550821" lastFinishedPulling="2026-06-02 20:04:47.557364988 +0000 UTC m=+163.754927671" observedRunningTime="2026-06-02 20:04:47.978976596 +0000 UTC m=+164.176539291" watchObservedRunningTime="2026-06-02 20:04:47.979984736 +0000 UTC m=+164.177547459" Jun 02 20:04:50.307800 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:50.307765 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx"] Jun 02 20:04:50.310084 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:50.310066 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" Jun 02 20:04:50.312140 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:50.312112 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qvhx2\"" Jun 02 20:04:50.312406 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:50.312388 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 02 20:04:50.320395 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:50.320372 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx"] Jun 02 20:04:50.433547 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:50.433518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/39ee9e2a-0684-4e09-86bf-2cdb77db149c-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-xwjgx\" (UID: \"39ee9e2a-0684-4e09-86bf-2cdb77db149c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" Jun 02 20:04:50.534063 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:50.534028 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/39ee9e2a-0684-4e09-86bf-2cdb77db149c-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-xwjgx\" (UID: \"39ee9e2a-0684-4e09-86bf-2cdb77db149c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" Jun 02 20:04:50.534226 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:50.534110 2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 02 20:04:50.534226 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:50.534178 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39ee9e2a-0684-4e09-86bf-2cdb77db149c-tls-certificates podName:39ee9e2a-0684-4e09-86bf-2cdb77db149c nodeName:}" failed. No retries permitted until 2026-06-02 20:04:51.034161204 +0000 UTC m=+167.231723877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/39ee9e2a-0684-4e09-86bf-2cdb77db149c-tls-certificates") pod "prometheus-operator-admission-webhook-59b7f8756b-xwjgx" (UID: "39ee9e2a-0684-4e09-86bf-2cdb77db149c") : secret "prometheus-operator-admission-webhook-tls" not found Jun 02 20:04:51.037936 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.037901 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/39ee9e2a-0684-4e09-86bf-2cdb77db149c-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-xwjgx\" (UID: \"39ee9e2a-0684-4e09-86bf-2cdb77db149c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" Jun 02 20:04:51.040345 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.040323 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/39ee9e2a-0684-4e09-86bf-2cdb77db149c-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-xwjgx\" (UID: \"39ee9e2a-0684-4e09-86bf-2cdb77db149c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" Jun 02 20:04:51.218809 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.218777 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" Jun 02 20:04:51.337314 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.337279 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx"] Jun 02 20:04:51.340290 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:51.340261 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ee9e2a_0684_4e09_86bf_2cdb77db149c.slice/crio-33a9c49dff1ee825033fe393ffc8873cb91d4333e536fee9d7a1e654ace36efb WatchSource:0}: Error finding container 33a9c49dff1ee825033fe393ffc8873cb91d4333e536fee9d7a1e654ace36efb: Status 404 returned error can't find the container with id 33a9c49dff1ee825033fe393ffc8873cb91d4333e536fee9d7a1e654ace36efb Jun 02 20:04:51.445794 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.445765 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5c6f989f-gh29v"] Jun 02 20:04:51.448539 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.448519 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.451232 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.451206 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 20:04:51.451332 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.451218 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 20:04:51.452048 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.452030 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zxf22\"" Jun 02 20:04:51.452048 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.452040 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 02 20:04:51.452226 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.452077 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 02 20:04:51.452226 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.452127 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 20:04:51.452226 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.452137 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 20:04:51.459808 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.459791 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 20:04:51.468747 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.468727 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5c6f989f-gh29v"] Jun 02 20:04:51.541103 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.541080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-serving-cert\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.541233 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.541110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-oauth-config\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.541233 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.541140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t4tp\" (UniqueName: \"kubernetes.io/projected/2535e4ca-ddf2-487c-992f-fa037ee5f31e-kube-api-access-4t4tp\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.541233 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.541164 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-config\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.541341 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.541284 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-service-ca\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.541341 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.541323 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-oauth-serving-cert\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.641841 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.641787 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4tp\" (UniqueName: \"kubernetes.io/projected/2535e4ca-ddf2-487c-992f-fa037ee5f31e-kube-api-access-4t4tp\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.641841 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.641815 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-config\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.641989 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.641969 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-service-ca\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.642056 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.642033 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-oauth-serving-cert\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.642108 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.642071 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-serving-cert\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.642108 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.642101 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-oauth-config\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.642453 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.642432 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-config\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.642621 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.642603 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-service-ca\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.642764 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.642744 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-oauth-serving-cert\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.644437 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.644412 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-oauth-config\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.644537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.644490 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-serving-cert\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.651976 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.651949 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t4tp\" (UniqueName: \"kubernetes.io/projected/2535e4ca-ddf2-487c-992f-fa037ee5f31e-kube-api-access-4t4tp\") pod \"console-5c5c6f989f-gh29v\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.758346 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.758316 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:04:51.878150 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.878117 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5c6f989f-gh29v"] Jun 02 20:04:51.880898 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:51.880870 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2535e4ca_ddf2_487c_992f_fa037ee5f31e.slice/crio-5983808a494238a209947dcc961e1fd40fee7822388873eb79a637e5adb8db70 WatchSource:0}: Error finding container 5983808a494238a209947dcc961e1fd40fee7822388873eb79a637e5adb8db70: Status 404 returned error can't find the container with id 5983808a494238a209947dcc961e1fd40fee7822388873eb79a637e5adb8db70 Jun 02 20:04:51.971110 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.971074 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" event={"ID":"39ee9e2a-0684-4e09-86bf-2cdb77db149c","Type":"ContainerStarted","Data":"33a9c49dff1ee825033fe393ffc8873cb91d4333e536fee9d7a1e654ace36efb"} Jun 02 20:04:51.971881 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:51.971859 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c6f989f-gh29v" event={"ID":"2535e4ca-ddf2-487c-992f-fa037ee5f31e","Type":"ContainerStarted","Data":"5983808a494238a209947dcc961e1fd40fee7822388873eb79a637e5adb8db70"} Jun 02 20:04:52.609078 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:52.609052 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:04:53.978461 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:53.978427 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" event={"ID":"39ee9e2a-0684-4e09-86bf-2cdb77db149c","Type":"ContainerStarted","Data":"8b58db514e5771ee9a100bcd5193016aff518191241af1de79c4b92a06291979"} Jun 02 20:04:53.978880 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:53.978638 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" Jun 02 20:04:53.984557 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:53.984533 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" Jun 02 20:04:54.000731 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.000684 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-xwjgx" podStartSLOduration=2.336557816 podStartE2EDuration="4.000668215s" podCreationTimestamp="2026-06-02 20:04:50 +0000 UTC" firstStartedPulling="2026-06-02 20:04:51.342243804 +0000 UTC m=+167.539806483" lastFinishedPulling="2026-06-02 20:04:53.006354208 +0000 UTC m=+169.203916882" observedRunningTime="2026-06-02 20:04:54.000424909 +0000 UTC m=+170.197987635" watchObservedRunningTime="2026-06-02 20:04:54.000668215 +0000 UTC m=+170.198230912" Jun 02 20:04:54.372795 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.372712 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s"] Jun 02 20:04:54.374906 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.374882 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.377514 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.377495 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 20:04:54.378061 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.378028 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 20:04:54.378061 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.378026 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 20:04:54.378218 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.378060 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 02 20:04:54.378218 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.378040 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 02 20:04:54.378218 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.378114 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-94l6h\"" Jun 02 20:04:54.386128 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.386109 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s"] Jun 02 20:04:54.438751 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.438722 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:04:54.465188 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.465154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.465338 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.465205 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrpww\" (UniqueName: \"kubernetes.io/projected/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-kube-api-access-rrpww\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.465338 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.465236 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.465338 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.465320 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.566292 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.566257 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.566460 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.566331 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.566460 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.566362 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.566460 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.566394 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rrpww\" (UniqueName: \"kubernetes.io/projected/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-kube-api-access-rrpww\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.567071 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.567044 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.568953 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.568931 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.569228 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.569195 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.576135 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.576096 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrpww\" (UniqueName: \"kubernetes.io/projected/8fc61135-d7b1-4fd1-9559-dbf7ec574e7c-kube-api-access-rrpww\") pod \"prometheus-operator-658bcd8bdf-55v4s\" (UID: \"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:54.684867 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:54.684839 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" Jun 02 20:04:55.410746 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:55.410717 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s"] Jun 02 20:04:55.414232 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:55.414208 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc61135_d7b1_4fd1_9559_dbf7ec574e7c.slice/crio-1e95038af2f7f3ff4a3b9440017d85bd4563c60d05b53fc9ffd8b74b5eb31cb1 WatchSource:0}: Error finding container 1e95038af2f7f3ff4a3b9440017d85bd4563c60d05b53fc9ffd8b74b5eb31cb1: Status 404 returned error can't find the container with id 1e95038af2f7f3ff4a3b9440017d85bd4563c60d05b53fc9ffd8b74b5eb31cb1 Jun 02 20:04:55.436144 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:55.436124 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:04:55.438531 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:55.438513 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hqb8x\"" Jun 02 20:04:55.447429 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:55.447408 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cdhs6" Jun 02 20:04:55.567302 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:55.567230 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cdhs6"] Jun 02 20:04:55.570853 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:04:55.570825 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f0a22d1_2b80_4b90_ba52_b598b876a693.slice/crio-29e196e1c42058a22137da586caf7ce3da7c41411d709ce379935aa06896784b WatchSource:0}: Error finding container 29e196e1c42058a22137da586caf7ce3da7c41411d709ce379935aa06896784b: Status 404 returned error can't find the container with id 29e196e1c42058a22137da586caf7ce3da7c41411d709ce379935aa06896784b Jun 02 20:04:55.984515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:55.984484 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" event={"ID":"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c","Type":"ContainerStarted","Data":"1e95038af2f7f3ff4a3b9440017d85bd4563c60d05b53fc9ffd8b74b5eb31cb1"} Jun 02 20:04:55.985815 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:55.985778 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c6f989f-gh29v" event={"ID":"2535e4ca-ddf2-487c-992f-fa037ee5f31e","Type":"ContainerStarted","Data":"eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7"} Jun 02 20:04:55.986801 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:55.986772 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cdhs6" event={"ID":"2f0a22d1-2b80-4b90-ba52-b598b876a693","Type":"ContainerStarted","Data":"29e196e1c42058a22137da586caf7ce3da7c41411d709ce379935aa06896784b"} Jun 02 20:04:56.004723 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:56.004680 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5c5c6f989f-gh29v" podStartSLOduration=1.560735486 podStartE2EDuration="5.004667877s" podCreationTimestamp="2026-06-02 20:04:51 +0000 UTC" firstStartedPulling="2026-06-02 20:04:51.882804727 +0000 UTC m=+168.080367400" lastFinishedPulling="2026-06-02 20:04:55.326737109 +0000 UTC m=+171.524299791" observedRunningTime="2026-06-02 20:04:56.003988098 +0000 UTC m=+172.201550792" watchObservedRunningTime="2026-06-02 20:04:56.004667877 +0000 UTC m=+172.202230571" Jun 02 20:04:57.965820 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:57.965792 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zkhfn" Jun 02 20:04:57.993841 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:57.993811 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" event={"ID":"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c","Type":"ContainerStarted","Data":"3e78065921f578f87f991730992428a0c28d197e842ea0a90842e86e215295bf"} Jun 02 20:04:57.994014 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:57.993849 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" event={"ID":"8fc61135-d7b1-4fd1-9559-dbf7ec574e7c","Type":"ContainerStarted","Data":"d331db40323561504b89998fea9e04cfead171ea1854c688aa0a6fe206816bc8"} Jun 02 20:04:57.995038 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:57.995017 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cdhs6" event={"ID":"2f0a22d1-2b80-4b90-ba52-b598b876a693","Type":"ContainerStarted","Data":"789347be0f6bd13e1cc935084e77166751033fee8099bded73c40dfcd9eb2e19"} Jun 02 20:04:58.017454 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:58.017403 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-55v4s" podStartSLOduration=1.986616951 podStartE2EDuration="4.017389161s" podCreationTimestamp="2026-06-02 20:04:54 +0000 UTC" firstStartedPulling="2026-06-02 20:04:55.416333831 +0000 UTC m=+171.613896503" lastFinishedPulling="2026-06-02 20:04:57.447106037 +0000 UTC m=+173.644668713" observedRunningTime="2026-06-02 20:04:58.017195627 +0000 UTC m=+174.214758339" watchObservedRunningTime="2026-06-02 20:04:58.017389161 +0000 UTC m=+174.214951855" Jun 02 20:04:58.055949 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:58.055907 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cdhs6" podStartSLOduration=139.177694131 podStartE2EDuration="2m21.055895254s" podCreationTimestamp="2026-06-02 20:02:37 +0000 UTC" firstStartedPulling="2026-06-02 20:04:55.572662907 +0000 UTC m=+171.770225581" lastFinishedPulling="2026-06-02 20:04:57.450864029 +0000 UTC m=+173.648426704" observedRunningTime="2026-06-02 20:04:58.05491985 +0000 UTC m=+174.252482544" watchObservedRunningTime="2026-06-02 20:04:58.055895254 +0000 UTC m=+174.253457948" Jun 02 20:04:59.730745 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.730711 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-kprxq"] Jun 02 20:04:59.732831 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.732813 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.735016 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.734974 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 02 20:04:59.735357 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.735341 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bw7qg\"" Jun 02 20:04:59.735650 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.735637 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 02 20:04:59.747360 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.747331 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-kprxq"] Jun 02 20:04:59.808274 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.808245 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.808274 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.808276 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.808447 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.808297 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.808447 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.808380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbx7f\" (UniqueName: \"kubernetes.io/projected/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-kube-api-access-pbx7f\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.813860 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.813835 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nctx4"] Jun 02 20:04:59.816356 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.816340 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.819391 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.819376 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 20:04:59.821277 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.821257 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 20:04:59.821497 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.821481 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gmdj2\"" Jun 02 20:04:59.821567 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.821505 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 20:04:59.909385 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz2g6\" (UniqueName: \"kubernetes.io/projected/deaba9a1-8f8c-4625-a280-2f3fc55ae934-kube-api-access-rz2g6\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.909537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909392 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pbx7f\" (UniqueName: \"kubernetes.io/projected/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-kube-api-access-pbx7f\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.909537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909441 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.909537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909460 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/deaba9a1-8f8c-4625-a280-2f3fc55ae934-root\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.909537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909481 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.909537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909500 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.909537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909528 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-accelerators-collector-config\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.909862 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:59.909595 2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 02 20:04:59.909862 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909649 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-textfile\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.909862 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:04:59.909663 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-openshift-state-metrics-tls podName:7e4ab581-ed5e-4429-a136-5c01fe9e4d26 nodeName:}" failed. No retries permitted until 2026-06-02 20:05:00.409641721 +0000 UTC m=+176.607204394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-openshift-state-metrics-tls") pod "openshift-state-metrics-85658796db-kprxq" (UID: "7e4ab581-ed5e-4429-a136-5c01fe9e4d26") : secret "openshift-state-metrics-tls" not found Jun 02 20:04:59.909862 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909705 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.909862 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909758 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-wtmp\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.909862 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909802 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/deaba9a1-8f8c-4625-a280-2f3fc55ae934-sys\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.909862 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909858 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/deaba9a1-8f8c-4625-a280-2f3fc55ae934-metrics-client-ca\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.910272 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.909892 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-tls\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:04:59.910329 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.910279 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.911861 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.911843 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:04:59.929783 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:04:59.929749 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbx7f\" (UniqueName: \"kubernetes.io/projected/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-kube-api-access-pbx7f\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:05:00.010921 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.010827 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/deaba9a1-8f8c-4625-a280-2f3fc55ae934-root\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.010921 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.010873 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-accelerators-collector-config\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.010921 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.010909 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-textfile\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.010923 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/deaba9a1-8f8c-4625-a280-2f3fc55ae934-root\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.010933 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011022 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-wtmp\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011065 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/deaba9a1-8f8c-4625-a280-2f3fc55ae934-sys\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011092 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/deaba9a1-8f8c-4625-a280-2f3fc55ae934-metrics-client-ca\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011123 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-tls\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011166 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/deaba9a1-8f8c-4625-a280-2f3fc55ae934-sys\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011175 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rz2g6\" (UniqueName: \"kubernetes.io/projected/deaba9a1-8f8c-4625-a280-2f3fc55ae934-kube-api-access-rz2g6\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011573 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011210 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-wtmp\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011573 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:05:00.011223 2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 20:05:00.011573 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:05:00.011278 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-tls podName:deaba9a1-8f8c-4625-a280-2f3fc55ae934 nodeName:}" failed. No retries permitted until 2026-06-02 20:05:00.511259922 +0000 UTC m=+176.708822595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-tls") pod "node-exporter-nctx4" (UID: "deaba9a1-8f8c-4625-a280-2f3fc55ae934") : secret "node-exporter-tls" not found Jun 02 20:05:00.011573 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011385 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-textfile\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011573 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011550 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-accelerators-collector-config\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.011827 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.011783 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/deaba9a1-8f8c-4625-a280-2f3fc55ae934-metrics-client-ca\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.013413 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.013395 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.024928 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.024904 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz2g6\" (UniqueName: \"kubernetes.io/projected/deaba9a1-8f8c-4625-a280-2f3fc55ae934-kube-api-access-rz2g6\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.415014 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.414965 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:05:00.417495 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.417470 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e4ab581-ed5e-4429-a136-5c01fe9e4d26-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-kprxq\" (UID: \"7e4ab581-ed5e-4429-a136-5c01fe9e4d26\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:05:00.516336 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.516299 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-tls\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.518615 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.518593 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/deaba9a1-8f8c-4625-a280-2f3fc55ae934-node-exporter-tls\") pod \"node-exporter-nctx4\" (UID: \"deaba9a1-8f8c-4625-a280-2f3fc55ae934\") " pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.641658 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.641622 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" Jun 02 20:05:00.725294 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.725267 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nctx4" Jun 02 20:05:00.735697 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:05:00.735664 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeaba9a1_8f8c_4625_a280_2f3fc55ae934.slice/crio-a2892c8de681aa054f3e52fc9229b7e43e69f8968ca6d1c347b9d4364d92d62b WatchSource:0}: Error finding container a2892c8de681aa054f3e52fc9229b7e43e69f8968ca6d1c347b9d4364d92d62b: Status 404 returned error can't find the container with id a2892c8de681aa054f3e52fc9229b7e43e69f8968ca6d1c347b9d4364d92d62b Jun 02 20:05:00.791395 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:00.791365 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-kprxq"] Jun 02 20:05:00.794159 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:05:00.794130 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4ab581_ed5e_4429_a136_5c01fe9e4d26.slice/crio-07316b6c8cf18d9b4ef2d0ad00c85bb9ad39387a43157689e3875725e22ff474 WatchSource:0}: Error finding container 07316b6c8cf18d9b4ef2d0ad00c85bb9ad39387a43157689e3875725e22ff474: Status 404 returned error can't find the container with id 07316b6c8cf18d9b4ef2d0ad00c85bb9ad39387a43157689e3875725e22ff474 Jun 02 20:05:01.004230 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:01.004148 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nctx4" event={"ID":"deaba9a1-8f8c-4625-a280-2f3fc55ae934","Type":"ContainerStarted","Data":"a2892c8de681aa054f3e52fc9229b7e43e69f8968ca6d1c347b9d4364d92d62b"} Jun 02 20:05:01.005949 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:01.005927 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" event={"ID":"7e4ab581-ed5e-4429-a136-5c01fe9e4d26","Type":"ContainerStarted","Data":"d4318a76d1388c093b6bac3e09af447cb85ea57140b45eda0a4d9a89d4aad505"} Jun 02 20:05:01.005949 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:01.005953 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" event={"ID":"7e4ab581-ed5e-4429-a136-5c01fe9e4d26","Type":"ContainerStarted","Data":"b37aa48574032a1773e93b69ce7719708214049951f131fc42bad0a4d3ea8a8e"} Jun 02 20:05:01.006120 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:01.005962 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" event={"ID":"7e4ab581-ed5e-4429-a136-5c01fe9e4d26","Type":"ContainerStarted","Data":"07316b6c8cf18d9b4ef2d0ad00c85bb9ad39387a43157689e3875725e22ff474"} Jun 02 20:05:01.758913 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:01.758879 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:05:01.759370 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:01.758974 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:05:01.764132 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:01.764101 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:05:02.013655 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:02.013605 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:05:03.013595 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:03.013559 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" event={"ID":"7e4ab581-ed5e-4429-a136-5c01fe9e4d26","Type":"ContainerStarted","Data":"5cb1cf99c1d48b3cc3be6f3b7718253711a0f92f55500e7a79bb521cbbc04ebb"} Jun 02 20:05:03.015076 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:03.015053 2577 generic.go:358] "Generic (PLEG): container finished" podID="deaba9a1-8f8c-4625-a280-2f3fc55ae934" containerID="c0e30f57fc846b30b08095aebc74ecd73cfd67f77b13a719d1dab18a11ccfdf0" exitCode=0 Jun 02 20:05:03.015181 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:03.015140 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nctx4" event={"ID":"deaba9a1-8f8c-4625-a280-2f3fc55ae934","Type":"ContainerDied","Data":"c0e30f57fc846b30b08095aebc74ecd73cfd67f77b13a719d1dab18a11ccfdf0"} Jun 02 20:05:03.035919 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:03.035866 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-kprxq" podStartSLOduration=2.9779364680000002 podStartE2EDuration="4.03584942s" podCreationTimestamp="2026-06-02 20:04:59 +0000 UTC" firstStartedPulling="2026-06-02 20:05:00.912652581 +0000 UTC m=+177.110215253" lastFinishedPulling="2026-06-02 20:05:01.970565532 +0000 UTC m=+178.168128205" observedRunningTime="2026-06-02 20:05:03.035106404 +0000 UTC m=+179.232669112" watchObservedRunningTime="2026-06-02 20:05:03.03584942 +0000 UTC m=+179.233412117" Jun 02 20:05:04.021319 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.021276 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nctx4" event={"ID":"deaba9a1-8f8c-4625-a280-2f3fc55ae934","Type":"ContainerStarted","Data":"7e0aef3c1ab04bbcf14765e5685d9d27639227c1282c2564cc86ba3bd6593f3f"} Jun 02 20:05:04.021319 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.021321 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nctx4" event={"ID":"deaba9a1-8f8c-4625-a280-2f3fc55ae934","Type":"ContainerStarted","Data":"6f5d70440aaeb1c0ca0546e5992a9e432cceb7be0ebbb7f186c9084c1f62ae70"} Jun 02 20:05:04.043761 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.043716 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nctx4" podStartSLOduration=3.812158808 podStartE2EDuration="5.043704113s" podCreationTimestamp="2026-06-02 20:04:59 +0000 UTC" firstStartedPulling="2026-06-02 20:05:00.737552341 +0000 UTC m=+176.935115014" lastFinishedPulling="2026-06-02 20:05:01.969097646 +0000 UTC m=+178.166660319" observedRunningTime="2026-06-02 20:05:04.041884108 +0000 UTC m=+180.239446815" watchObservedRunningTime="2026-06-02 20:05:04.043704113 +0000 UTC m=+180.241266808" Jun 02 20:05:04.248792 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.248756 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6ff98f5777-89rld"] Jun 02 20:05:04.251311 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.251292 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.253892 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.253873 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 20:05:04.253987 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.253873 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 02 20:05:04.254387 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.254363 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 02 20:05:04.254501 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.254393 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2jc7m\"" Jun 02 20:05:04.254501 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.254404 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 02 20:05:04.254501 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.254447 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cpb1dse3elk7r\"" Jun 02 20:05:04.264307 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.264286 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6ff98f5777-89rld"] Jun 02 20:05:04.351066 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.350979 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4phn7\" (UniqueName: \"kubernetes.io/projected/c987865a-3e14-4421-914f-9bbefd1b113c-kube-api-access-4phn7\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.351066 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.351029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c987865a-3e14-4421-914f-9bbefd1b113c-audit-log\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.351066 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.351058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c987865a-3e14-4421-914f-9bbefd1b113c-secret-metrics-server-client-certs\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.351261 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.351076 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c987865a-3e14-4421-914f-9bbefd1b113c-metrics-server-audit-profiles\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.351261 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.351121 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c987865a-3e14-4421-914f-9bbefd1b113c-secret-metrics-server-tls\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.351261 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.351163 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c987865a-3e14-4421-914f-9bbefd1b113c-client-ca-bundle\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.351261 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.351192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c987865a-3e14-4421-914f-9bbefd1b113c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.452192 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.452161 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c987865a-3e14-4421-914f-9bbefd1b113c-secret-metrics-server-tls\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.452331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.452197 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c987865a-3e14-4421-914f-9bbefd1b113c-client-ca-bundle\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.452331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.452217 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c987865a-3e14-4421-914f-9bbefd1b113c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.452331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.452248 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4phn7\" (UniqueName: \"kubernetes.io/projected/c987865a-3e14-4421-914f-9bbefd1b113c-kube-api-access-4phn7\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.452331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.452266 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c987865a-3e14-4421-914f-9bbefd1b113c-audit-log\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.452331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.452289 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c987865a-3e14-4421-914f-9bbefd1b113c-secret-metrics-server-client-certs\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.452331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.452305 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c987865a-3e14-4421-914f-9bbefd1b113c-metrics-server-audit-profiles\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.452646 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.452627 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c987865a-3e14-4421-914f-9bbefd1b113c-audit-log\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.454475 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.454458 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 02 20:05:04.454596 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.454579 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 20:05:04.454639 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.454611 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 02 20:05:04.454709 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.454691 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cpb1dse3elk7r\"" Jun 02 20:05:04.454770 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.454758 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 02 20:05:04.463605 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.463579 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c987865a-3e14-4421-914f-9bbefd1b113c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.463945 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.463905 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c987865a-3e14-4421-914f-9bbefd1b113c-metrics-server-audit-profiles\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.464159 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.464136 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4phn7\" (UniqueName: \"kubernetes.io/projected/c987865a-3e14-4421-914f-9bbefd1b113c-kube-api-access-4phn7\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.465770 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.465747 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c987865a-3e14-4421-914f-9bbefd1b113c-client-ca-bundle\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.466054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.465988 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c987865a-3e14-4421-914f-9bbefd1b113c-secret-metrics-server-client-certs\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.467049 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.467016 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c987865a-3e14-4421-914f-9bbefd1b113c-secret-metrics-server-tls\") pod \"metrics-server-6ff98f5777-89rld\" (UID: \"c987865a-3e14-4421-914f-9bbefd1b113c\") " pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.563429 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.563398 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2jc7m\"" Jun 02 20:05:04.571945 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.571921 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:04.696495 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:04.696463 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6ff98f5777-89rld"] Jun 02 20:05:04.699169 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:05:04.699143 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc987865a_3e14_4421_914f_9bbefd1b113c.slice/crio-f253964f83b2a385bf0d927b5b017324e6a3967f205c7a5b3ad6ca8b792139c6 WatchSource:0}: Error finding container f253964f83b2a385bf0d927b5b017324e6a3967f205c7a5b3ad6ca8b792139c6: Status 404 returned error can't find the container with id f253964f83b2a385bf0d927b5b017324e6a3967f205c7a5b3ad6ca8b792139c6 Jun 02 20:05:05.024882 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:05.024839 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" event={"ID":"c987865a-3e14-4421-914f-9bbefd1b113c","Type":"ContainerStarted","Data":"f253964f83b2a385bf0d927b5b017324e6a3967f205c7a5b3ad6ca8b792139c6"} Jun 02 20:05:06.059269 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.059229 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 20:05:06.062964 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.062936 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.066010 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.065827 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 20:05:06.066010 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.065941 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 20:05:06.066184 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.066059 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 20:05:06.066239 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.066194 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 20:05:06.066239 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.066213 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 20:05:06.066387 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.066367 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-g4x2v\"" Jun 02 20:05:06.066800 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.066663 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 20:05:06.066800 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.066682 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 20:05:06.066958 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.066819 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d2d1lib352o2n\"" Jun 02 20:05:06.067046 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.067026 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 20:05:06.067486 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.067027 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 20:05:06.067486 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.067239 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 20:05:06.067486 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.067376 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 20:05:06.070872 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.070853 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 20:05:06.077658 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.077613 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 20:05:06.169126 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169314 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169135 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169314 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169174 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169314 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169261 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169314 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169299 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169514 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169326 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169514 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169514 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169514 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169514 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169498 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169771 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169563 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169771 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7m2x\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-kube-api-access-x7m2x\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169771 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169637 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-config\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169771 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.169771 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169745 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.170027 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169787 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.170027 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169811 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.170027 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.169857 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271646 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271703 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271734 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271761 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271795 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271822 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271849 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271888 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271922 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x7m2x\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-kube-api-access-x7m2x\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271947 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-config\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.271971 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272836 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.272372 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272836 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.272428 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272836 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.272458 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272836 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.272485 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272836 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.272529 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272836 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.272556 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.272836 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.272597 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.273217 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.273048 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.273470 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.273441 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.275400 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.275306 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.277021 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.276360 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.277021 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.276411 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.277021 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.276874 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-web-config\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.277021 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.276939 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.277378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.277355 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.277460 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.277397 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-config\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.277552 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.277528 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.277613 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.277566 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.277850 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.277824 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-config-out\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.278098 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.278052 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.278384 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.278359 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.279436 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.279408 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.279921 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.279897 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.280165 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.280145 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.281273 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.281253 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7m2x\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-kube-api-access-x7m2x\") pod \"prometheus-k8s-0\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.377737 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.377704 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:06.540032 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:06.539967 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 20:05:06.542271 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:05:06.542239 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a39f341_5c66_4551_90f8_0923838c5ae3.slice/crio-3a3e6392e57ff6460e11067e6dca98ae8dccf1c2eeaa8e2fef176dad59ae7d3c WatchSource:0}: Error finding container 3a3e6392e57ff6460e11067e6dca98ae8dccf1c2eeaa8e2fef176dad59ae7d3c: Status 404 returned error can't find the container with id 3a3e6392e57ff6460e11067e6dca98ae8dccf1c2eeaa8e2fef176dad59ae7d3c Jun 02 20:05:07.031630 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.031589 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" event={"ID":"c987865a-3e14-4421-914f-9bbefd1b113c","Type":"ContainerStarted","Data":"92db4694dc2d6cb59a3229e82b07369cee40f5cbec06730b6bc3d244c9036915"} Jun 02 20:05:07.032648 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.032620 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerStarted","Data":"3a3e6392e57ff6460e11067e6dca98ae8dccf1c2eeaa8e2fef176dad59ae7d3c"} Jun 02 20:05:07.050460 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.050416 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" podStartSLOduration=1.44063002 podStartE2EDuration="3.050398127s" podCreationTimestamp="2026-06-02 20:05:04 +0000 UTC" firstStartedPulling="2026-06-02 20:05:04.701131614 +0000 UTC m=+180.898694299" lastFinishedPulling="2026-06-02 20:05:06.31089972 +0000 UTC m=+182.508462406" observedRunningTime="2026-06-02 20:05:07.048777207 +0000 UTC m=+183.246339925" watchObservedRunningTime="2026-06-02 20:05:07.050398127 +0000 UTC m=+183.247960824" Jun 02 20:05:07.624271 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.624179 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" podUID="949edd5e-089b-4737-8e96-335fc4a3e472" containerName="registry" containerID="cri-o://37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04" gracePeriod=30 Jun 02 20:05:07.863065 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.863040 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:05:07.989974 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.989944 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgl89\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-kube-api-access-qgl89\") pod \"949edd5e-089b-4737-8e96-335fc4a3e472\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " Jun 02 20:05:07.990143 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.989981 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-image-registry-private-configuration\") pod \"949edd5e-089b-4737-8e96-335fc4a3e472\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " Jun 02 20:05:07.990143 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.990020 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-bound-sa-token\") pod \"949edd5e-089b-4737-8e96-335fc4a3e472\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " Jun 02 20:05:07.990143 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.990041 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-trusted-ca\") pod \"949edd5e-089b-4737-8e96-335fc4a3e472\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " Jun 02 20:05:07.990143 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.990099 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") pod \"949edd5e-089b-4737-8e96-335fc4a3e472\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " Jun 02 20:05:07.990143 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.990134 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949edd5e-089b-4737-8e96-335fc4a3e472-ca-trust-extracted\") pod \"949edd5e-089b-4737-8e96-335fc4a3e472\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " Jun 02 20:05:07.990317 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.990159 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-registry-certificates\") pod \"949edd5e-089b-4737-8e96-335fc4a3e472\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " Jun 02 20:05:07.990317 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.990219 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-installation-pull-secrets\") pod \"949edd5e-089b-4737-8e96-335fc4a3e472\" (UID: \"949edd5e-089b-4737-8e96-335fc4a3e472\") " Jun 02 20:05:07.990784 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.990581 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "949edd5e-089b-4737-8e96-335fc4a3e472" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:05:07.990784 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.990713 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "949edd5e-089b-4737-8e96-335fc4a3e472" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:05:07.992438 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.992413 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "949edd5e-089b-4737-8e96-335fc4a3e472" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:05:07.992748 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.992724 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-kube-api-access-qgl89" (OuterVolumeSpecName: "kube-api-access-qgl89") pod "949edd5e-089b-4737-8e96-335fc4a3e472" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472"). InnerVolumeSpecName "kube-api-access-qgl89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:05:07.992810 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.992743 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "949edd5e-089b-4737-8e96-335fc4a3e472" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:05:07.992810 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.992759 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "949edd5e-089b-4737-8e96-335fc4a3e472" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:05:07.992810 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.992791 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "949edd5e-089b-4737-8e96-335fc4a3e472" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:05:07.998432 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:07.998410 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949edd5e-089b-4737-8e96-335fc4a3e472-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "949edd5e-089b-4737-8e96-335fc4a3e472" (UID: "949edd5e-089b-4737-8e96-335fc4a3e472"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 20:05:08.037258 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.037215 2577 generic.go:358] "Generic (PLEG): container finished" podID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerID="80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23" exitCode=0 Jun 02 20:05:08.037371 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.037305 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerDied","Data":"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23"} Jun 02 20:05:08.038430 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.038405 2577 generic.go:358] "Generic (PLEG): container finished" podID="949edd5e-089b-4737-8e96-335fc4a3e472" containerID="37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04" exitCode=0 Jun 02 20:05:08.038556 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.038467 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" Jun 02 20:05:08.038556 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.038494 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" event={"ID":"949edd5e-089b-4737-8e96-335fc4a3e472","Type":"ContainerDied","Data":"37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04"} Jun 02 20:05:08.038556 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.038519 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf56fffcf-glffz" event={"ID":"949edd5e-089b-4737-8e96-335fc4a3e472","Type":"ContainerDied","Data":"190066732bd9fd421949a21ed8aeb0f19dc8c31a238d34625267951f0c5b81c9"} Jun 02 20:05:08.038556 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.038540 2577 scope.go:117] "RemoveContainer" containerID="37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04" Jun 02 20:05:08.046833 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.046791 2577 scope.go:117] "RemoveContainer" containerID="37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04" Jun 02 20:05:08.047125 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:05:08.047102 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04\": container with ID starting with 37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04 not found: ID does not exist" containerID="37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04" Jun 02 20:05:08.047188 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.047136 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04"} err="failed to get container status \"37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04\": rpc error: code = NotFound desc = could not find container \"37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04\": container with ID starting with 37d642e4e0c779437781dcf7c41e5f954aadbc2ba7aeb7392647bb9383379e04 not found: ID does not exist" Jun 02 20:05:08.090826 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.090798 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/949edd5e-089b-4737-8e96-335fc4a3e472-ca-trust-extracted\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:08.090826 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.090823 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-registry-certificates\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:08.090952 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.090832 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-installation-pull-secrets\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:08.090952 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.090841 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgl89\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-kube-api-access-qgl89\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:08.090952 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.090851 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/949edd5e-089b-4737-8e96-335fc4a3e472-image-registry-private-configuration\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:08.090952 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.090860 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-bound-sa-token\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:08.090952 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.090869 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/949edd5e-089b-4737-8e96-335fc4a3e472-trusted-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:08.090952 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.090877 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/949edd5e-089b-4737-8e96-335fc4a3e472-registry-tls\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:08.110108 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.110056 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cf56fffcf-glffz"] Jun 02 20:05:08.120982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.120960 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cf56fffcf-glffz"] Jun 02 20:05:08.440809 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:08.440776 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="949edd5e-089b-4737-8e96-335fc4a3e472" path="/var/lib/kubelet/pods/949edd5e-089b-4737-8e96-335fc4a3e472/volumes" Jun 02 20:05:11.049935 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:11.049851 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerStarted","Data":"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c"} Jun 02 20:05:11.049935 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:11.049889 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerStarted","Data":"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8"} Jun 02 20:05:13.059568 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:13.059525 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerStarted","Data":"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a"} Jun 02 20:05:13.059568 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:13.059572 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerStarted","Data":"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4"} Jun 02 20:05:13.059969 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:13.059586 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerStarted","Data":"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea"} Jun 02 20:05:13.059969 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:13.059599 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerStarted","Data":"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d"} Jun 02 20:05:13.088494 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:13.088444 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=0.93147585 podStartE2EDuration="7.088430077s" podCreationTimestamp="2026-06-02 20:05:06 +0000 UTC" firstStartedPulling="2026-06-02 20:05:06.544182263 +0000 UTC m=+182.741744938" lastFinishedPulling="2026-06-02 20:05:12.701136485 +0000 UTC m=+188.898699165" observedRunningTime="2026-06-02 20:05:13.086418996 +0000 UTC m=+189.283981690" watchObservedRunningTime="2026-06-02 20:05:13.088430077 +0000 UTC m=+189.285992810" Jun 02 20:05:16.378011 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:16.377952 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:05:17.528931 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:17.528894 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5c6f989f-gh29v"] Jun 02 20:05:24.572193 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:24.572161 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:24.572193 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:24.572195 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:34.117054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:34.117021 2577 generic.go:358] "Generic (PLEG): container finished" podID="0b220d09-f221-4b6f-935e-58a4d367abd3" containerID="7c57bb075c5ada2a38af35f1026c4039488dc31476ebeeac666d9a63248630cf" exitCode=0 Jun 02 20:05:34.117455 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:34.117081 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" event={"ID":"0b220d09-f221-4b6f-935e-58a4d367abd3","Type":"ContainerDied","Data":"7c57bb075c5ada2a38af35f1026c4039488dc31476ebeeac666d9a63248630cf"} Jun 02 20:05:34.117455 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:34.117404 2577 scope.go:117] "RemoveContainer" containerID="7c57bb075c5ada2a38af35f1026c4039488dc31476ebeeac666d9a63248630cf" Jun 02 20:05:35.122405 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:35.122373 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-twl68" event={"ID":"0b220d09-f221-4b6f-935e-58a4d367abd3","Type":"ContainerStarted","Data":"de41611ffd17182c4303c964e851f9238c9790e2a232b91454f485181c38c2c2"} Jun 02 20:05:42.551427 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.551380 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5c5c6f989f-gh29v" podUID="2535e4ca-ddf2-487c-992f-fa037ee5f31e" containerName="console" containerID="cri-o://eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7" gracePeriod=15 Jun 02 20:05:42.801158 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.801136 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c5c6f989f-gh29v_2535e4ca-ddf2-487c-992f-fa037ee5f31e/console/0.log" Jun 02 20:05:42.801268 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.801195 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:05:42.882621 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.882593 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-serving-cert\") pod \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " Jun 02 20:05:42.882621 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.882631 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-service-ca\") pod \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " Jun 02 20:05:42.882829 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.882656 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t4tp\" (UniqueName: \"kubernetes.io/projected/2535e4ca-ddf2-487c-992f-fa037ee5f31e-kube-api-access-4t4tp\") pod \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " Jun 02 20:05:42.882829 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.882717 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-config\") pod \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " Jun 02 20:05:42.882829 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.882745 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-oauth-serving-cert\") pod \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " Jun 02 20:05:42.882829 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.882788 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-oauth-config\") pod \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\" (UID: \"2535e4ca-ddf2-487c-992f-fa037ee5f31e\") " Jun 02 20:05:42.883155 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.883117 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2535e4ca-ddf2-487c-992f-fa037ee5f31e" (UID: "2535e4ca-ddf2-487c-992f-fa037ee5f31e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:05:42.883294 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.883212 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-config" (OuterVolumeSpecName: "console-config") pod "2535e4ca-ddf2-487c-992f-fa037ee5f31e" (UID: "2535e4ca-ddf2-487c-992f-fa037ee5f31e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:05:42.883294 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.883248 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-service-ca" (OuterVolumeSpecName: "service-ca") pod "2535e4ca-ddf2-487c-992f-fa037ee5f31e" (UID: "2535e4ca-ddf2-487c-992f-fa037ee5f31e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:05:42.884984 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.884959 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2535e4ca-ddf2-487c-992f-fa037ee5f31e" (UID: "2535e4ca-ddf2-487c-992f-fa037ee5f31e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:05:42.885083 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.884968 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2535e4ca-ddf2-487c-992f-fa037ee5f31e-kube-api-access-4t4tp" (OuterVolumeSpecName: "kube-api-access-4t4tp") pod "2535e4ca-ddf2-487c-992f-fa037ee5f31e" (UID: "2535e4ca-ddf2-487c-992f-fa037ee5f31e"). InnerVolumeSpecName "kube-api-access-4t4tp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:05:42.885083 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.884986 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2535e4ca-ddf2-487c-992f-fa037ee5f31e" (UID: "2535e4ca-ddf2-487c-992f-fa037ee5f31e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:05:42.983899 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.983822 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-serving-cert\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:42.983899 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.983852 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:42.983899 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.983862 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t4tp\" (UniqueName: \"kubernetes.io/projected/2535e4ca-ddf2-487c-992f-fa037ee5f31e-kube-api-access-4t4tp\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:42.983899 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.983872 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-config\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:42.983899 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.983881 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2535e4ca-ddf2-487c-992f-fa037ee5f31e-oauth-serving-cert\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:42.983899 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:42.983889 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2535e4ca-ddf2-487c-992f-fa037ee5f31e-console-oauth-config\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:05:43.143816 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.143789 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c5c6f989f-gh29v_2535e4ca-ddf2-487c-992f-fa037ee5f31e/console/0.log" Jun 02 20:05:43.143975 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.143826 2577 generic.go:358] "Generic (PLEG): container finished" podID="2535e4ca-ddf2-487c-992f-fa037ee5f31e" containerID="eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7" exitCode=2 Jun 02 20:05:43.143975 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.143861 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c6f989f-gh29v" event={"ID":"2535e4ca-ddf2-487c-992f-fa037ee5f31e","Type":"ContainerDied","Data":"eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7"} Jun 02 20:05:43.143975 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.143903 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5c6f989f-gh29v" event={"ID":"2535e4ca-ddf2-487c-992f-fa037ee5f31e","Type":"ContainerDied","Data":"5983808a494238a209947dcc961e1fd40fee7822388873eb79a637e5adb8db70"} Jun 02 20:05:43.143975 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.143919 2577 scope.go:117] "RemoveContainer" containerID="eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7" Jun 02 20:05:43.143975 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.143921 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5c6f989f-gh29v" Jun 02 20:05:43.151913 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.151896 2577 scope.go:117] "RemoveContainer" containerID="eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7" Jun 02 20:05:43.152175 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:05:43.152156 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7\": container with ID starting with eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7 not found: ID does not exist" containerID="eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7" Jun 02 20:05:43.152229 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.152182 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7"} err="failed to get container status \"eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7\": rpc error: code = NotFound desc = could not find container \"eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7\": container with ID starting with eade673821fb907f44f4ddb2e2e2802eec2c72ede3610df0c1cbf2485b0328c7 not found: ID does not exist" Jun 02 20:05:43.164707 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.164683 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5c6f989f-gh29v"] Jun 02 20:05:43.168341 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:43.168322 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5c6f989f-gh29v"] Jun 02 20:05:44.440255 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:44.440225 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2535e4ca-ddf2-487c-992f-fa037ee5f31e" path="/var/lib/kubelet/pods/2535e4ca-ddf2-487c-992f-fa037ee5f31e/volumes" Jun 02 20:05:44.577710 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:44.577681 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:05:44.581431 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:05:44.581411 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6ff98f5777-89rld" Jun 02 20:06:06.377886 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:06.377841 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:06.397789 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:06.397763 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:07.224353 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:07.224327 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:15.146623 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:15.146585 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:06:15.148831 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:15.148811 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d34cf515-d3ea-4298-97e9-3b98e1963700-metrics-certs\") pod \"network-metrics-daemon-q8xf6\" (UID: \"d34cf515-d3ea-4298-97e9-3b98e1963700\") " pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:06:15.441696 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:15.441664 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s8fv6\"" Jun 02 20:06:15.449881 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:15.449858 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8xf6" Jun 02 20:06:15.573485 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:15.573443 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q8xf6"] Jun 02 20:06:15.576064 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:06:15.576035 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34cf515_d3ea_4298_97e9_3b98e1963700.slice/crio-4fd7207ed986dc4df0645aa980268144290201149c54e982ec0c63d1d940ebf6 WatchSource:0}: Error finding container 4fd7207ed986dc4df0645aa980268144290201149c54e982ec0c63d1d940ebf6: Status 404 returned error can't find the container with id 4fd7207ed986dc4df0645aa980268144290201149c54e982ec0c63d1d940ebf6 Jun 02 20:06:16.234978 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:16.234937 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q8xf6" event={"ID":"d34cf515-d3ea-4298-97e9-3b98e1963700","Type":"ContainerStarted","Data":"4fd7207ed986dc4df0645aa980268144290201149c54e982ec0c63d1d940ebf6"} Jun 02 20:06:17.240691 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:17.240649 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q8xf6" event={"ID":"d34cf515-d3ea-4298-97e9-3b98e1963700","Type":"ContainerStarted","Data":"b66f6ea5cdc5731738f2ef81414625b92521fc2eb482ccb1ea5d63282f82ad45"} Jun 02 20:06:17.241064 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:17.240699 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q8xf6" event={"ID":"d34cf515-d3ea-4298-97e9-3b98e1963700","Type":"ContainerStarted","Data":"aef5e7c820dcbd68bcc6af365d4641a516832d896dcb42e5dd4b1fef3ffd74b4"} Jun 02 20:06:17.257923 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:17.257870 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-q8xf6" podStartSLOduration=252.135832451 podStartE2EDuration="4m13.257855205s" podCreationTimestamp="2026-06-02 20:02:04 +0000 UTC" firstStartedPulling="2026-06-02 20:06:15.578144685 +0000 UTC m=+251.775707372" lastFinishedPulling="2026-06-02 20:06:16.70016744 +0000 UTC m=+252.897730126" observedRunningTime="2026-06-02 20:06:17.256913135 +0000 UTC m=+253.454475828" watchObservedRunningTime="2026-06-02 20:06:17.257855205 +0000 UTC m=+253.455417893" Jun 02 20:06:24.412752 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.412663 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 20:06:24.413267 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.413228 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="prometheus" containerID="cri-o://d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" gracePeriod=600 Jun 02 20:06:24.413432 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.413354 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy-web" containerID="cri-o://00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" gracePeriod=600 Jun 02 20:06:24.413432 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.413355 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="config-reloader" containerID="cri-o://d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" gracePeriod=600 Jun 02 20:06:24.413572 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.413276 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy" containerID="cri-o://f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" gracePeriod=600 Jun 02 20:06:24.413572 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.413433 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy-thanos" containerID="cri-o://95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" gracePeriod=600 Jun 02 20:06:24.413572 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.413534 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="thanos-sidecar" containerID="cri-o://05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" gracePeriod=600 Jun 02 20:06:24.654960 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.654936 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:24.724681 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.724594 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-config\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.724681 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.724639 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-metrics-client-ca\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.724681 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.724673 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-tls\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725056 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-kube-rbac-proxy\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725112 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7m2x\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-kube-api-access-x7m2x\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725125 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725150 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725179 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-config-out\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725209 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-kubelet-serving-ca-bundle\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725236 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-thanos-prometheus-http-client-file\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725273 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-tls-assets\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725301 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-grpc-tls\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725354 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-db\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725378 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-rulefiles-0\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725413 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-web-config\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725443 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-serving-certs-ca-bundle\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725492 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-trusted-ca-bundle\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725531 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.725545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725560 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-metrics-client-certs\") pod \"8a39f341-5c66-4551-90f8-0923838c5ae3\" (UID: \"8a39f341-5c66-4551-90f8-0923838c5ae3\") " Jun 02 20:06:24.726537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.725815 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-metrics-client-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.726537 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.726144 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:06:24.726643 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.726588 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:06:24.731365 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.731072 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:06:24.731365 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.731174 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:06:24.731365 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.731287 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-kube-api-access-x7m2x" (OuterVolumeSpecName: "kube-api-access-x7m2x") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "kube-api-access-x7m2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:06:24.731706 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.731546 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-config" (OuterVolumeSpecName: "config") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:06:24.731898 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.731712 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:06:24.732789 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.732757 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 20:06:24.733060 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.733029 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:06:24.734190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.734165 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-config-out" (OuterVolumeSpecName: "config-out") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 20:06:24.735151 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.735126 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:06:24.740240 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.740212 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:06:24.740240 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.740201 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:06:24.741549 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.740576 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 20:06:24.741726 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.741705 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:06:24.741883 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.741854 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:06:24.749455 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.749434 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-web-config" (OuterVolumeSpecName: "web-config") pod "8a39f341-5c66-4551-90f8-0923838c5ae3" (UID: "8a39f341-5c66-4551-90f8-0923838c5ae3"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 20:06:24.827236 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827197 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-db\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827236 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827233 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827236 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827244 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-web-config\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827255 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827264 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827275 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827285 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-metrics-client-certs\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827294 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-config\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827302 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827311 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-kube-rbac-proxy\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827320 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7m2x\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-kube-api-access-x7m2x\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827328 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827337 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8a39f341-5c66-4551-90f8-0923838c5ae3-config-out\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827345 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a39f341-5c66-4551-90f8-0923838c5ae3-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827354 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827363 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8a39f341-5c66-4551-90f8-0923838c5ae3-tls-assets\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:24.827464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:24.827371 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a39f341-5c66-4551-90f8-0923838c5ae3-secret-grpc-tls\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:06:25.269508 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269474 2577 generic.go:358] "Generic (PLEG): container finished" podID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerID="95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" exitCode=0 Jun 02 20:06:25.269508 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269503 2577 generic.go:358] "Generic (PLEG): container finished" podID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerID="f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" exitCode=0 Jun 02 20:06:25.269508 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269510 2577 generic.go:358] "Generic (PLEG): container finished" podID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerID="00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" exitCode=0 Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269518 2577 generic.go:358] "Generic (PLEG): container finished" podID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerID="05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" exitCode=0 Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269526 2577 generic.go:358] "Generic (PLEG): container finished" podID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerID="d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" exitCode=0 Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269534 2577 generic.go:358] "Generic (PLEG): container finished" podID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerID="d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" exitCode=0 Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269551 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerDied","Data":"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a"} Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269593 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerDied","Data":"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4"} Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269597 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269607 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerDied","Data":"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea"} Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269616 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerDied","Data":"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d"} Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269626 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerDied","Data":"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c"} Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269635 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerDied","Data":"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8"} Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269648 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8a39f341-5c66-4551-90f8-0923838c5ae3","Type":"ContainerDied","Data":"3a3e6392e57ff6460e11067e6dca98ae8dccf1c2eeaa8e2fef176dad59ae7d3c"} Jun 02 20:06:25.269725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.269663 2577 scope.go:117] "RemoveContainer" containerID="95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" Jun 02 20:06:25.277832 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.277816 2577 scope.go:117] "RemoveContainer" containerID="f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" Jun 02 20:06:25.284461 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.284443 2577 scope.go:117] "RemoveContainer" containerID="00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" Jun 02 20:06:25.290353 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.290337 2577 scope.go:117] "RemoveContainer" containerID="05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" Jun 02 20:06:25.292672 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.292653 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 20:06:25.297153 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.297126 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 20:06:25.297575 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.297554 2577 scope.go:117] "RemoveContainer" containerID="d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" Jun 02 20:06:25.305497 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.305465 2577 scope.go:117] "RemoveContainer" containerID="d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" Jun 02 20:06:25.313280 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.313259 2577 scope.go:117] "RemoveContainer" containerID="80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23" Jun 02 20:06:25.319467 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.319450 2577 scope.go:117] "RemoveContainer" containerID="95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" Jun 02 20:06:25.319718 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:06:25.319699 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": container with ID starting with 95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a not found: ID does not exist" containerID="95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" Jun 02 20:06:25.319758 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.319727 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a"} err="failed to get container status \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": rpc error: code = NotFound desc = could not find container \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": container with ID starting with 95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a not found: ID does not exist" Jun 02 20:06:25.319758 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.319746 2577 scope.go:117] "RemoveContainer" containerID="f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" Jun 02 20:06:25.319963 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:06:25.319947 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": container with ID starting with f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4 not found: ID does not exist" containerID="f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" Jun 02 20:06:25.320031 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.319968 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4"} err="failed to get container status \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": rpc error: code = NotFound desc = could not find container \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": container with ID starting with f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4 not found: ID does not exist" Jun 02 20:06:25.320031 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.319983 2577 scope.go:117] "RemoveContainer" containerID="00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" Jun 02 20:06:25.320225 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:06:25.320210 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": container with ID starting with 00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea not found: ID does not exist" containerID="00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" Jun 02 20:06:25.320265 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.320229 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea"} err="failed to get container status \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": rpc error: code = NotFound desc = could not find container \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": container with ID starting with 00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea not found: ID does not exist" Jun 02 20:06:25.320265 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.320242 2577 scope.go:117] "RemoveContainer" containerID="05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" Jun 02 20:06:25.320460 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:06:25.320444 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": container with ID starting with 05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d not found: ID does not exist" containerID="05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" Jun 02 20:06:25.320494 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.320465 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d"} err="failed to get container status \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": rpc error: code = NotFound desc = could not find container \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": container with ID starting with 05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d not found: ID does not exist" Jun 02 20:06:25.320494 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.320477 2577 scope.go:117] "RemoveContainer" containerID="d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" Jun 02 20:06:25.320676 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:06:25.320659 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": container with ID starting with d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c not found: ID does not exist" containerID="d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" Jun 02 20:06:25.320716 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.320681 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c"} err="failed to get container status \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": rpc error: code = NotFound desc = could not find container \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": container with ID starting with d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c not found: ID does not exist" Jun 02 20:06:25.320716 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.320695 2577 scope.go:117] "RemoveContainer" containerID="d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" Jun 02 20:06:25.320907 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:06:25.320891 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": container with ID starting with d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8 not found: ID does not exist" containerID="d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" Jun 02 20:06:25.320947 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.320914 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8"} err="failed to get container status \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": rpc error: code = NotFound desc = could not find container \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": container with ID starting with d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8 not found: ID does not exist" Jun 02 20:06:25.320947 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.320928 2577 scope.go:117] "RemoveContainer" containerID="80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23" Jun 02 20:06:25.321179 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:06:25.321164 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": container with ID starting with 80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23 not found: ID does not exist" containerID="80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23" Jun 02 20:06:25.321219 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321183 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23"} err="failed to get container status \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": rpc error: code = NotFound desc = could not find container \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": container with ID starting with 80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23 not found: ID does not exist" Jun 02 20:06:25.321219 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321195 2577 scope.go:117] "RemoveContainer" containerID="95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" Jun 02 20:06:25.321408 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321391 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a"} err="failed to get container status \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": rpc error: code = NotFound desc = could not find container \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": container with ID starting with 95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a not found: ID does not exist" Jun 02 20:06:25.321447 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321409 2577 scope.go:117] "RemoveContainer" containerID="f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" Jun 02 20:06:25.321597 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321582 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4"} err="failed to get container status \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": rpc error: code = NotFound desc = could not find container \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": container with ID starting with f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4 not found: ID does not exist" Jun 02 20:06:25.321644 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321597 2577 scope.go:117] "RemoveContainer" containerID="00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" Jun 02 20:06:25.321778 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321761 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea"} err="failed to get container status \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": rpc error: code = NotFound desc = could not find container \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": container with ID starting with 00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea not found: ID does not exist" Jun 02 20:06:25.321824 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321778 2577 scope.go:117] "RemoveContainer" containerID="05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" Jun 02 20:06:25.321977 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321959 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d"} err="failed to get container status \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": rpc error: code = NotFound desc = could not find container \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": container with ID starting with 05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d not found: ID does not exist" Jun 02 20:06:25.322038 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.321977 2577 scope.go:117] "RemoveContainer" containerID="d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" Jun 02 20:06:25.322219 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322198 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c"} err="failed to get container status \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": rpc error: code = NotFound desc = could not find container \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": container with ID starting with d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c not found: ID does not exist" Jun 02 20:06:25.322294 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322221 2577 scope.go:117] "RemoveContainer" containerID="d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" Jun 02 20:06:25.322420 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322403 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8"} err="failed to get container status \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": rpc error: code = NotFound desc = could not find container \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": container with ID starting with d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8 not found: ID does not exist" Jun 02 20:06:25.322470 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322421 2577 scope.go:117] "RemoveContainer" containerID="80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23" Jun 02 20:06:25.322609 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322590 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23"} err="failed to get container status \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": rpc error: code = NotFound desc = could not find container \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": container with ID starting with 80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23 not found: ID does not exist" Jun 02 20:06:25.322675 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322610 2577 scope.go:117] "RemoveContainer" containerID="95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" Jun 02 20:06:25.322806 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322788 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a"} err="failed to get container status \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": rpc error: code = NotFound desc = could not find container \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": container with ID starting with 95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a not found: ID does not exist" Jun 02 20:06:25.322858 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322806 2577 scope.go:117] "RemoveContainer" containerID="f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" Jun 02 20:06:25.322972 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322958 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4"} err="failed to get container status \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": rpc error: code = NotFound desc = could not find container \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": container with ID starting with f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4 not found: ID does not exist" Jun 02 20:06:25.322972 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.322971 2577 scope.go:117] "RemoveContainer" containerID="00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" Jun 02 20:06:25.323203 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323186 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea"} err="failed to get container status \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": rpc error: code = NotFound desc = could not find container \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": container with ID starting with 00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea not found: ID does not exist" Jun 02 20:06:25.323254 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323204 2577 scope.go:117] "RemoveContainer" containerID="05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" Jun 02 20:06:25.323377 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323352 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d"} err="failed to get container status \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": rpc error: code = NotFound desc = could not find container \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": container with ID starting with 05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d not found: ID does not exist" Jun 02 20:06:25.323462 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323378 2577 scope.go:117] "RemoveContainer" containerID="d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" Jun 02 20:06:25.323555 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323537 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c"} err="failed to get container status \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": rpc error: code = NotFound desc = could not find container \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": container with ID starting with d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c not found: ID does not exist" Jun 02 20:06:25.323555 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323554 2577 scope.go:117] "RemoveContainer" containerID="d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" Jun 02 20:06:25.323734 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323720 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8"} err="failed to get container status \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": rpc error: code = NotFound desc = could not find container \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": container with ID starting with d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8 not found: ID does not exist" Jun 02 20:06:25.323734 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323733 2577 scope.go:117] "RemoveContainer" containerID="80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23" Jun 02 20:06:25.323941 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323920 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23"} err="failed to get container status \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": rpc error: code = NotFound desc = could not find container \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": container with ID starting with 80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23 not found: ID does not exist" Jun 02 20:06:25.323941 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.323941 2577 scope.go:117] "RemoveContainer" containerID="95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" Jun 02 20:06:25.324192 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324173 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a"} err="failed to get container status \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": rpc error: code = NotFound desc = could not find container \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": container with ID starting with 95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a not found: ID does not exist" Jun 02 20:06:25.324234 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324192 2577 scope.go:117] "RemoveContainer" containerID="f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" Jun 02 20:06:25.324392 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324374 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4"} err="failed to get container status \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": rpc error: code = NotFound desc = could not find container \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": container with ID starting with f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4 not found: ID does not exist" Jun 02 20:06:25.324438 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324392 2577 scope.go:117] "RemoveContainer" containerID="00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" Jun 02 20:06:25.324573 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324558 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea"} err="failed to get container status \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": rpc error: code = NotFound desc = could not find container \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": container with ID starting with 00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea not found: ID does not exist" Jun 02 20:06:25.324615 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324573 2577 scope.go:117] "RemoveContainer" containerID="05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" Jun 02 20:06:25.324760 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324745 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d"} err="failed to get container status \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": rpc error: code = NotFound desc = could not find container \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": container with ID starting with 05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d not found: ID does not exist" Jun 02 20:06:25.324798 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324759 2577 scope.go:117] "RemoveContainer" containerID="d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" Jun 02 20:06:25.324950 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324933 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c"} err="failed to get container status \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": rpc error: code = NotFound desc = could not find container \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": container with ID starting with d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c not found: ID does not exist" Jun 02 20:06:25.325012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.324952 2577 scope.go:117] "RemoveContainer" containerID="d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" Jun 02 20:06:25.325161 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325143 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8"} err="failed to get container status \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": rpc error: code = NotFound desc = could not find container \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": container with ID starting with d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8 not found: ID does not exist" Jun 02 20:06:25.325205 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325161 2577 scope.go:117] "RemoveContainer" containerID="80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23" Jun 02 20:06:25.325334 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325316 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23"} err="failed to get container status \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": rpc error: code = NotFound desc = could not find container \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": container with ID starting with 80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23 not found: ID does not exist" Jun 02 20:06:25.325377 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325337 2577 scope.go:117] "RemoveContainer" containerID="95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" Jun 02 20:06:25.325555 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325538 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a"} err="failed to get container status \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": rpc error: code = NotFound desc = could not find container \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": container with ID starting with 95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a not found: ID does not exist" Jun 02 20:06:25.325597 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325555 2577 scope.go:117] "RemoveContainer" containerID="f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" Jun 02 20:06:25.325751 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325729 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4"} err="failed to get container status \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": rpc error: code = NotFound desc = could not find container \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": container with ID starting with f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4 not found: ID does not exist" Jun 02 20:06:25.325814 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325752 2577 scope.go:117] "RemoveContainer" containerID="00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" Jun 02 20:06:25.325933 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325917 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea"} err="failed to get container status \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": rpc error: code = NotFound desc = could not find container \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": container with ID starting with 00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea not found: ID does not exist" Jun 02 20:06:25.325979 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.325933 2577 scope.go:117] "RemoveContainer" containerID="05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" Jun 02 20:06:25.326128 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326113 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d"} err="failed to get container status \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": rpc error: code = NotFound desc = could not find container \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": container with ID starting with 05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d not found: ID does not exist" Jun 02 20:06:25.326179 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326128 2577 scope.go:117] "RemoveContainer" containerID="d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" Jun 02 20:06:25.326329 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326313 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c"} err="failed to get container status \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": rpc error: code = NotFound desc = could not find container \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": container with ID starting with d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c not found: ID does not exist" Jun 02 20:06:25.326329 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326327 2577 scope.go:117] "RemoveContainer" containerID="d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" Jun 02 20:06:25.326514 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326496 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8"} err="failed to get container status \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": rpc error: code = NotFound desc = could not find container \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": container with ID starting with d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8 not found: ID does not exist" Jun 02 20:06:25.326561 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326516 2577 scope.go:117] "RemoveContainer" containerID="80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23" Jun 02 20:06:25.326709 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326693 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23"} err="failed to get container status \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": rpc error: code = NotFound desc = could not find container \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": container with ID starting with 80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23 not found: ID does not exist" Jun 02 20:06:25.326753 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326709 2577 scope.go:117] "RemoveContainer" containerID="95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a" Jun 02 20:06:25.326983 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326958 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a"} err="failed to get container status \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": rpc error: code = NotFound desc = could not find container \"95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a\": container with ID starting with 95345a96a364339b6cef78d288a6ab914d59a75e7a1f9a721a8cac7b29e9ce6a not found: ID does not exist" Jun 02 20:06:25.327122 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.326984 2577 scope.go:117] "RemoveContainer" containerID="f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4" Jun 02 20:06:25.327285 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.327253 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4"} err="failed to get container status \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": rpc error: code = NotFound desc = could not find container \"f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4\": container with ID starting with f62bb0b2b50fd3bf4fba0f9a8eae20feee9e5329fc39259b81387673d30319a4 not found: ID does not exist" Jun 02 20:06:25.327376 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.327286 2577 scope.go:117] "RemoveContainer" containerID="00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea" Jun 02 20:06:25.327584 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.327555 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea"} err="failed to get container status \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": rpc error: code = NotFound desc = could not find container \"00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea\": container with ID starting with 00d9c8f3800418e0d0dca656261b0f8037543faa49b7efd0b8f69aaa42e252ea not found: ID does not exist" Jun 02 20:06:25.327665 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.327585 2577 scope.go:117] "RemoveContainer" containerID="05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d" Jun 02 20:06:25.327882 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.327851 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d"} err="failed to get container status \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": rpc error: code = NotFound desc = could not find container \"05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d\": container with ID starting with 05e0a2f4a866b2be279cda6ea5cd3c8fdea50cd04971842118a0d0f1072e897d not found: ID does not exist" Jun 02 20:06:25.327882 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.327873 2577 scope.go:117] "RemoveContainer" containerID="d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c" Jun 02 20:06:25.328157 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.328115 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c"} err="failed to get container status \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": rpc error: code = NotFound desc = could not find container \"d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c\": container with ID starting with d44b0110ba1f7049bc1756f16ab7f9ecc70c9afe4dafe371ae711e72f0fc9c4c not found: ID does not exist" Jun 02 20:06:25.328157 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.328154 2577 scope.go:117] "RemoveContainer" containerID="d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8" Jun 02 20:06:25.328407 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.328380 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8"} err="failed to get container status \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": rpc error: code = NotFound desc = could not find container \"d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8\": container with ID starting with d37da877c5890a29edca033ae08fc224d6523a4097ff1af347e72f46c44eb5f8 not found: ID does not exist" Jun 02 20:06:25.328464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.328407 2577 scope.go:117] "RemoveContainer" containerID="80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23" Jun 02 20:06:25.328647 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.328623 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23"} err="failed to get container status \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": rpc error: code = NotFound desc = could not find container \"80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23\": container with ID starting with 80bf894a878e90d16a0145f1609eb32ca85c12b14f45d87829252ca52f8c3b23 not found: ID does not exist" Jun 02 20:06:25.329369 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329347 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 20:06:25.329639 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329627 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="config-reloader" Jun 02 20:06:25.329688 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329641 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="config-reloader" Jun 02 20:06:25.329688 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329648 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="thanos-sidecar" Jun 02 20:06:25.329688 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329656 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="thanos-sidecar" Jun 02 20:06:25.329688 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329677 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949edd5e-089b-4737-8e96-335fc4a3e472" containerName="registry" Jun 02 20:06:25.329688 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329688 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="949edd5e-089b-4737-8e96-335fc4a3e472" containerName="registry" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329697 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2535e4ca-ddf2-487c-992f-fa037ee5f31e" containerName="console" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329703 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2535e4ca-ddf2-487c-992f-fa037ee5f31e" containerName="console" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329711 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="prometheus" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329716 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="prometheus" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329724 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="init-config-reloader" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329729 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="init-config-reloader" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329735 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy-web" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329742 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy-web" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329751 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329757 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329763 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy-thanos" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329768 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy-thanos" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329813 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="thanos-sidecar" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329826 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329833 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="prometheus" Jun 02 20:06:25.329849 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329848 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="949edd5e-089b-4737-8e96-335fc4a3e472" containerName="registry" Jun 02 20:06:25.330291 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329854 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy-thanos" Jun 02 20:06:25.330291 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329866 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="2535e4ca-ddf2-487c-992f-fa037ee5f31e" containerName="console" Jun 02 20:06:25.330291 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329872 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="config-reloader" Jun 02 20:06:25.330291 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.329878 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" containerName="kube-rbac-proxy-web" Jun 02 20:06:25.334067 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.334052 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.338090 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.338070 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 20:06:25.338195 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.338174 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 20:06:25.338274 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.338199 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 20:06:25.338348 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.338329 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 20:06:25.338847 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.338831 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 20:06:25.338923 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.338865 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-g4x2v\"" Jun 02 20:06:25.339012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.338949 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 20:06:25.339012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.338981 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 20:06:25.339263 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.339244 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d2d1lib352o2n\"" Jun 02 20:06:25.339516 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.339498 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 20:06:25.340014 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.339975 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 20:06:25.342119 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.342094 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 20:06:25.342650 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.342633 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 20:06:25.345933 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.345917 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 20:06:25.349173 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.349143 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 20:06:25.433435 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433410 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-config-out\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433451 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433484 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433523 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433620 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433638 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-web-config\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgsws\" (UniqueName: \"kubernetes.io/projected/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-kube-api-access-hgsws\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433696 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433722 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.433755 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433741 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.434155 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433779 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.434155 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433800 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.434155 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433828 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.434155 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433845 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.434155 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433861 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-config\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.434155 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433878 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.434155 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.433929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535285 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535195 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535285 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535246 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-config\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535285 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535276 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535580 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535303 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535580 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535349 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-config-out\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535580 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535377 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535580 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535415 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535580 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535445 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535580 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535470 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535861 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535641 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535861 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535667 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-web-config\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535861 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535709 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hgsws\" (UniqueName: \"kubernetes.io/projected/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-kube-api-access-hgsws\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535861 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535741 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535861 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535773 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535861 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535801 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535861 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535826 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.535861 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535852 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.536280 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535876 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.536280 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.535878 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.536280 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.536229 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.536416 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.536313 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.537740 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.537356 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.538648 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.538624 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-config\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.538904 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.538880 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-config-out\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.539159 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.539135 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.539360 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.539319 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.539921 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.539620 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.540149 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.540118 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.540218 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.540185 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.540285 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.540212 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.540681 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.540657 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.540758 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.540738 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.540887 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.540868 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.541615 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.541588 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.541916 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.541899 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-web-config\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.544255 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.544238 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgsws\" (UniqueName: \"kubernetes.io/projected/4df56e78-cbea-4eb8-b0c5-472f8626cfd7-kube-api-access-hgsws\") pod \"prometheus-k8s-0\" (UID: \"4df56e78-cbea-4eb8-b0c5-472f8626cfd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.643904 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.643874 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:06:25.777631 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:25.777603 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 20:06:25.780158 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:06:25.780134 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df56e78_cbea_4eb8_b0c5_472f8626cfd7.slice/crio-6cdbbb4e075d8279fdbe1688d64d26cd68b7ce082f8374ce8e7c281f44f10d31 WatchSource:0}: Error finding container 6cdbbb4e075d8279fdbe1688d64d26cd68b7ce082f8374ce8e7c281f44f10d31: Status 404 returned error can't find the container with id 6cdbbb4e075d8279fdbe1688d64d26cd68b7ce082f8374ce8e7c281f44f10d31 Jun 02 20:06:26.274262 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:26.274230 2577 generic.go:358] "Generic (PLEG): container finished" podID="4df56e78-cbea-4eb8-b0c5-472f8626cfd7" containerID="00e8711edf28e13a99202e021eafa0b0ea1a27d980be4a9f43cc391c22cb8640" exitCode=0 Jun 02 20:06:26.274427 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:26.274308 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4df56e78-cbea-4eb8-b0c5-472f8626cfd7","Type":"ContainerDied","Data":"00e8711edf28e13a99202e021eafa0b0ea1a27d980be4a9f43cc391c22cb8640"} Jun 02 20:06:26.274427 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:26.274328 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4df56e78-cbea-4eb8-b0c5-472f8626cfd7","Type":"ContainerStarted","Data":"6cdbbb4e075d8279fdbe1688d64d26cd68b7ce082f8374ce8e7c281f44f10d31"} Jun 02 20:06:26.447134 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:26.447099 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a39f341-5c66-4551-90f8-0923838c5ae3" path="/var/lib/kubelet/pods/8a39f341-5c66-4551-90f8-0923838c5ae3/volumes" Jun 02 20:06:27.280557 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:27.280523 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4df56e78-cbea-4eb8-b0c5-472f8626cfd7","Type":"ContainerStarted","Data":"f1b6ea27ae62745483348eede9fd4941331a69eec43802e4ed1e036264942025"} Jun 02 20:06:27.280557 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:27.280559 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4df56e78-cbea-4eb8-b0c5-472f8626cfd7","Type":"ContainerStarted","Data":"6c6207e598b598d13c13243cbd81610e52fbaa7ecc6a4ae4d6ede745e501ca3f"} Jun 02 20:06:27.280774 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:27.280572 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4df56e78-cbea-4eb8-b0c5-472f8626cfd7","Type":"ContainerStarted","Data":"eab2931ba7563a703539875681f84947821c1f174101022517465f5f37185cdf"} Jun 02 20:06:27.280774 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:27.280583 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4df56e78-cbea-4eb8-b0c5-472f8626cfd7","Type":"ContainerStarted","Data":"e28e9aa5f053e634ce00ac4be375c697c342801271a3973473f21297df578d49"} Jun 02 20:06:27.280774 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:27.280594 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4df56e78-cbea-4eb8-b0c5-472f8626cfd7","Type":"ContainerStarted","Data":"1b889108700d07efb966a8b948b86b945e888e8c4cf237e5a6726f311025b76d"} Jun 02 20:06:27.280774 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:27.280608 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4df56e78-cbea-4eb8-b0c5-472f8626cfd7","Type":"ContainerStarted","Data":"f9fccf39a7ecc315b37236aae653bff4212fb9418a3ca83d0ca82fe5ab3b697a"} Jun 02 20:06:27.310300 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:27.310251 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.310236802 podStartE2EDuration="2.310236802s" podCreationTimestamp="2026-06-02 20:06:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:06:27.308292124 +0000 UTC m=+263.505854818" watchObservedRunningTime="2026-06-02 20:06:27.310236802 +0000 UTC m=+263.507799493" Jun 02 20:06:30.644311 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:06:30.644272 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:07:04.312765 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:07:04.312737 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:07:04.313276 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:07:04.313074 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:07:04.322114 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:07:04.322091 2577 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 20:07:25.644733 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:07:25.644699 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:07:25.660471 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:07:25.660445 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:07:26.463435 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:07:26.463407 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 20:09:19.972245 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:19.972212 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-j89vj"] Jun 02 20:09:19.974220 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:19.974204 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" Jun 02 20:09:19.976471 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:19.976448 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 02 20:09:19.977353 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:19.977337 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-w7p6s\"" Jun 02 20:09:19.977446 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:19.977376 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 02 20:09:19.983968 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:19.983943 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-j89vj"] Jun 02 20:09:20.111462 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.111426 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eff9d72a-8f1c-4b7c-b439-20bcc6c1c787-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-j89vj\" (UID: \"eff9d72a-8f1c-4b7c-b439-20bcc6c1c787\") " pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" Jun 02 20:09:20.111633 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.111482 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtpp4\" (UniqueName: \"kubernetes.io/projected/eff9d72a-8f1c-4b7c-b439-20bcc6c1c787-kube-api-access-mtpp4\") pod \"cert-manager-cainjector-68b757865b-j89vj\" (UID: \"eff9d72a-8f1c-4b7c-b439-20bcc6c1c787\") " pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" Jun 02 20:09:20.212568 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.212530 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mtpp4\" (UniqueName: \"kubernetes.io/projected/eff9d72a-8f1c-4b7c-b439-20bcc6c1c787-kube-api-access-mtpp4\") pod \"cert-manager-cainjector-68b757865b-j89vj\" (UID: \"eff9d72a-8f1c-4b7c-b439-20bcc6c1c787\") " pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" Jun 02 20:09:20.212794 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.212614 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eff9d72a-8f1c-4b7c-b439-20bcc6c1c787-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-j89vj\" (UID: \"eff9d72a-8f1c-4b7c-b439-20bcc6c1c787\") " pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" Jun 02 20:09:20.223016 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.222947 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eff9d72a-8f1c-4b7c-b439-20bcc6c1c787-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-j89vj\" (UID: \"eff9d72a-8f1c-4b7c-b439-20bcc6c1c787\") " pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" Jun 02 20:09:20.223190 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.223166 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtpp4\" (UniqueName: \"kubernetes.io/projected/eff9d72a-8f1c-4b7c-b439-20bcc6c1c787-kube-api-access-mtpp4\") pod \"cert-manager-cainjector-68b757865b-j89vj\" (UID: \"eff9d72a-8f1c-4b7c-b439-20bcc6c1c787\") " pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" Jun 02 20:09:20.295591 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.295566 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" Jun 02 20:09:20.417908 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.417877 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-j89vj"] Jun 02 20:09:20.420741 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:09:20.420714 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeff9d72a_8f1c_4b7c_b439_20bcc6c1c787.slice/crio-9abc697996d88df87f1bf5f15009ba13c028ba910da1b1207736409f9f50481d WatchSource:0}: Error finding container 9abc697996d88df87f1bf5f15009ba13c028ba910da1b1207736409f9f50481d: Status 404 returned error can't find the container with id 9abc697996d88df87f1bf5f15009ba13c028ba910da1b1207736409f9f50481d Jun 02 20:09:20.422471 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.422455 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 20:09:20.752280 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:20.752246 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" event={"ID":"eff9d72a-8f1c-4b7c-b439-20bcc6c1c787","Type":"ContainerStarted","Data":"9abc697996d88df87f1bf5f15009ba13c028ba910da1b1207736409f9f50481d"} Jun 02 20:09:22.701244 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.701206 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-8whfm"] Jun 02 20:09:22.705136 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.705113 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:09:22.707566 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.707543 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-l4zxt\"" Jun 02 20:09:22.714637 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.714611 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-8whfm"] Jun 02 20:09:22.835024 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.834931 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ef5ed69-2374-47ae-9541-68f23fa7782e-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-8whfm\" (UID: \"2ef5ed69-2374-47ae-9541-68f23fa7782e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:09:22.835024 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.834965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt447\" (UniqueName: \"kubernetes.io/projected/2ef5ed69-2374-47ae-9541-68f23fa7782e-kube-api-access-mt447\") pod \"cert-manager-webhook-587ccfb98-8whfm\" (UID: \"2ef5ed69-2374-47ae-9541-68f23fa7782e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:09:22.936070 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.936029 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ef5ed69-2374-47ae-9541-68f23fa7782e-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-8whfm\" (UID: \"2ef5ed69-2374-47ae-9541-68f23fa7782e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:09:22.936238 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.936079 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mt447\" (UniqueName: \"kubernetes.io/projected/2ef5ed69-2374-47ae-9541-68f23fa7782e-kube-api-access-mt447\") pod \"cert-manager-webhook-587ccfb98-8whfm\" (UID: \"2ef5ed69-2374-47ae-9541-68f23fa7782e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:09:22.945846 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.945818 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ef5ed69-2374-47ae-9541-68f23fa7782e-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-8whfm\" (UID: \"2ef5ed69-2374-47ae-9541-68f23fa7782e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:09:22.946131 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:22.946106 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt447\" (UniqueName: \"kubernetes.io/projected/2ef5ed69-2374-47ae-9541-68f23fa7782e-kube-api-access-mt447\") pod \"cert-manager-webhook-587ccfb98-8whfm\" (UID: \"2ef5ed69-2374-47ae-9541-68f23fa7782e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:09:23.016941 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:23.016821 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:09:23.831975 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:23.831933 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-8whfm"] Jun 02 20:09:23.836177 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:09:23.836148 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ef5ed69_2374_47ae_9541_68f23fa7782e.slice/crio-c898d92c79616c09b9900ffefeef6bbe74c2941941a02ca48cc852ddf9571ef9 WatchSource:0}: Error finding container c898d92c79616c09b9900ffefeef6bbe74c2941941a02ca48cc852ddf9571ef9: Status 404 returned error can't find the container with id c898d92c79616c09b9900ffefeef6bbe74c2941941a02ca48cc852ddf9571ef9 Jun 02 20:09:24.765239 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:24.765201 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" event={"ID":"2ef5ed69-2374-47ae-9541-68f23fa7782e","Type":"ContainerStarted","Data":"c2c416c3c588e8c0bdffe7879db3b5d75e63a845ff8dbd36288a8ac019b0899b"} Jun 02 20:09:24.765239 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:24.765244 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" event={"ID":"2ef5ed69-2374-47ae-9541-68f23fa7782e","Type":"ContainerStarted","Data":"c898d92c79616c09b9900ffefeef6bbe74c2941941a02ca48cc852ddf9571ef9"} Jun 02 20:09:24.765456 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:24.765294 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:09:24.766417 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:24.766398 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" event={"ID":"eff9d72a-8f1c-4b7c-b439-20bcc6c1c787","Type":"ContainerStarted","Data":"4a05bdeba98c572ee96f4dba72ac7a1a1d1135981df228c0cac9e14bdf8df5f7"} Jun 02 20:09:24.788362 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:24.788319 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" podStartSLOduration=2.788305868 podStartE2EDuration="2.788305868s" podCreationTimestamp="2026-06-02 20:09:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:09:24.787703587 +0000 UTC m=+440.985266282" watchObservedRunningTime="2026-06-02 20:09:24.788305868 +0000 UTC m=+440.985868562" Jun 02 20:09:24.805253 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:24.805205 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-j89vj" podStartSLOduration=2.496058187 podStartE2EDuration="5.805167597s" podCreationTimestamp="2026-06-02 20:09:19 +0000 UTC" firstStartedPulling="2026-06-02 20:09:20.42257849 +0000 UTC m=+436.620141166" lastFinishedPulling="2026-06-02 20:09:23.731687894 +0000 UTC m=+439.929250576" observedRunningTime="2026-06-02 20:09:24.80465798 +0000 UTC m=+441.002220675" watchObservedRunningTime="2026-06-02 20:09:24.805167597 +0000 UTC m=+441.002730294" Jun 02 20:09:30.771152 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:09:30.771119 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-8whfm" Jun 02 20:10:01.592311 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.592278 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f89f75957-rndbj"] Jun 02 20:10:01.597112 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.597089 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.603980 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.603564 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 20:10:01.603980 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.603580 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gvngf\"" Jun 02 20:10:01.603980 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.603603 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 20:10:01.603980 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.603610 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 20:10:01.603980 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.603571 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 20:10:01.606435 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.606416 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 20:10:01.608248 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.608228 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f89f75957-rndbj"] Jun 02 20:10:01.742571 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.742541 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq9rr\" (UniqueName: \"kubernetes.io/projected/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-kube-api-access-mq9rr\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.742737 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.742591 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-cert\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.742737 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.742657 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-manager-config\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.742811 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.742740 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-metrics-cert\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.843174 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.843103 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-manager-config\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.843174 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.843148 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-metrics-cert\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.843407 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.843319 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mq9rr\" (UniqueName: \"kubernetes.io/projected/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-kube-api-access-mq9rr\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.843407 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.843353 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-cert\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.843677 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.843650 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-manager-config\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.845488 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.845465 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-metrics-cert\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.845593 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.845576 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-cert\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.859428 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.859409 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq9rr\" (UniqueName: \"kubernetes.io/projected/bb1d5e4a-9184-4235-ad94-d4fee8ba93cf-kube-api-access-mq9rr\") pod \"lws-controller-manager-f89f75957-rndbj\" (UID: \"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf\") " pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:01.907063 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:01.907020 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:02.048158 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:02.048055 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f89f75957-rndbj"] Jun 02 20:10:02.050666 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:10:02.050638 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1d5e4a_9184_4235_ad94_d4fee8ba93cf.slice/crio-63beccf559d404a01f1e08f65b91ae387ec5d26ad63bdadbb833c69cfb3cb5cb WatchSource:0}: Error finding container 63beccf559d404a01f1e08f65b91ae387ec5d26ad63bdadbb833c69cfb3cb5cb: Status 404 returned error can't find the container with id 63beccf559d404a01f1e08f65b91ae387ec5d26ad63bdadbb833c69cfb3cb5cb Jun 02 20:10:02.873330 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:02.873295 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" event={"ID":"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf","Type":"ContainerStarted","Data":"63beccf559d404a01f1e08f65b91ae387ec5d26ad63bdadbb833c69cfb3cb5cb"} Jun 02 20:10:04.093123 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.093080 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn"] Jun 02 20:10:04.096326 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.096310 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.100964 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.100945 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-998mr\"" Jun 02 20:10:04.104007 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.103974 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 20:10:04.104184 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.104168 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 20:10:04.104304 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.104287 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 20:10:04.109619 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.109602 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 20:10:04.142325 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.142302 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn"] Jun 02 20:10:04.161149 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.161121 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/853d9549-b2d8-4c4b-98b4-750872603e09-manifests\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.161266 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.161169 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/853d9549-b2d8-4c4b-98b4-750872603e09-tmp\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.161266 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.161189 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5x7m\" (UniqueName: \"kubernetes.io/projected/853d9549-b2d8-4c4b-98b4-750872603e09-kube-api-access-f5x7m\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.161266 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.161249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/853d9549-b2d8-4c4b-98b4-750872603e09-webhook-cert\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.161378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.161279 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/853d9549-b2d8-4c4b-98b4-750872603e09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.262505 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.262476 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/853d9549-b2d8-4c4b-98b4-750872603e09-manifests\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.262618 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.262522 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/853d9549-b2d8-4c4b-98b4-750872603e09-tmp\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.262618 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.262545 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f5x7m\" (UniqueName: \"kubernetes.io/projected/853d9549-b2d8-4c4b-98b4-750872603e09-kube-api-access-f5x7m\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.262618 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.262586 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/853d9549-b2d8-4c4b-98b4-750872603e09-webhook-cert\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.262728 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.262620 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/853d9549-b2d8-4c4b-98b4-750872603e09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.262984 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.262958 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/853d9549-b2d8-4c4b-98b4-750872603e09-manifests\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.264817 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.264798 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/853d9549-b2d8-4c4b-98b4-750872603e09-tmp\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.264978 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.264962 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/853d9549-b2d8-4c4b-98b4-750872603e09-webhook-cert\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.265159 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.265139 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/853d9549-b2d8-4c4b-98b4-750872603e09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.286612 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.286590 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 20:10:04.297784 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.297764 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 20:10:04.306042 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.306017 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5x7m\" (UniqueName: \"kubernetes.io/projected/853d9549-b2d8-4c4b-98b4-750872603e09-kube-api-access-f5x7m\") pod \"opendatahub-operator-controller-manager-58b87b6c86-hjbdn\" (UID: \"853d9549-b2d8-4c4b-98b4-750872603e09\") " pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.409156 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.409090 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-998mr\"" Jun 02 20:10:04.417072 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.417054 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:04.578232 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.578157 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn"] Jun 02 20:10:04.581479 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:10:04.581449 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853d9549_b2d8_4c4b_98b4_750872603e09.slice/crio-441c90d758b8a7930e9c2af8e5fb54daf1ab41ea1d35fdd311c6bdbab1beb336 WatchSource:0}: Error finding container 441c90d758b8a7930e9c2af8e5fb54daf1ab41ea1d35fdd311c6bdbab1beb336: Status 404 returned error can't find the container with id 441c90d758b8a7930e9c2af8e5fb54daf1ab41ea1d35fdd311c6bdbab1beb336 Jun 02 20:10:04.882819 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:04.882769 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" event={"ID":"853d9549-b2d8-4c4b-98b4-750872603e09","Type":"ContainerStarted","Data":"441c90d758b8a7930e9c2af8e5fb54daf1ab41ea1d35fdd311c6bdbab1beb336"} Jun 02 20:10:08.902101 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:08.902055 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" event={"ID":"bb1d5e4a-9184-4235-ad94-d4fee8ba93cf","Type":"ContainerStarted","Data":"2849e7040b762da2163eb47615fa3f22efa2cf03b3d4d7095cb747af9492fe6f"} Jun 02 20:10:08.902533 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:08.902224 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:08.903493 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:08.903471 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" event={"ID":"853d9549-b2d8-4c4b-98b4-750872603e09","Type":"ContainerStarted","Data":"faeac2f88ad8313bafd223d87fea95a6ca2c7fd059fc660c6049c41d90ff1fe7"} Jun 02 20:10:08.929542 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:08.929494 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" podStartSLOduration=2.121885984 podStartE2EDuration="7.929479989s" podCreationTimestamp="2026-06-02 20:10:01 +0000 UTC" firstStartedPulling="2026-06-02 20:10:02.052432261 +0000 UTC m=+478.249994934" lastFinishedPulling="2026-06-02 20:10:07.860026263 +0000 UTC m=+484.057588939" observedRunningTime="2026-06-02 20:10:08.927976265 +0000 UTC m=+485.125538961" watchObservedRunningTime="2026-06-02 20:10:08.929479989 +0000 UTC m=+485.127042683" Jun 02 20:10:09.908419 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:09.908377 2577 generic.go:358] "Generic (PLEG): container finished" podID="853d9549-b2d8-4c4b-98b4-750872603e09" containerID="faeac2f88ad8313bafd223d87fea95a6ca2c7fd059fc660c6049c41d90ff1fe7" exitCode=0 Jun 02 20:10:09.908841 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:09.908472 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" event={"ID":"853d9549-b2d8-4c4b-98b4-750872603e09","Type":"ContainerDied","Data":"faeac2f88ad8313bafd223d87fea95a6ca2c7fd059fc660c6049c41d90ff1fe7"} Jun 02 20:10:10.913625 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:10.913590 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" event={"ID":"853d9549-b2d8-4c4b-98b4-750872603e09","Type":"ContainerStarted","Data":"81e0e5fb9b3443864f2f748fe227525d56bd598dcda482ca1356ad43b771b647"} Jun 02 20:10:10.914012 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:10.913751 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:10.936742 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:10.936699 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" podStartSLOduration=1.224638151 podStartE2EDuration="6.936686093s" podCreationTimestamp="2026-06-02 20:10:04 +0000 UTC" firstStartedPulling="2026-06-02 20:10:04.584243676 +0000 UTC m=+480.781806352" lastFinishedPulling="2026-06-02 20:10:10.296291621 +0000 UTC m=+486.493854294" observedRunningTime="2026-06-02 20:10:10.934530095 +0000 UTC m=+487.132092814" watchObservedRunningTime="2026-06-02 20:10:10.936686093 +0000 UTC m=+487.134248788" Jun 02 20:10:19.910982 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:19.910951 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f89f75957-rndbj" Jun 02 20:10:21.919273 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:21.919238 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58b87b6c86-hjbdn" Jun 02 20:10:24.334526 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.334493 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk"] Jun 02 20:10:24.337905 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.337883 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.341112 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.341091 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-l72rs\"" Jun 02 20:10:24.341458 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.341442 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 02 20:10:24.341869 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.341849 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 02 20:10:24.341977 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.341958 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 20:10:24.342055 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.342036 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 20:10:24.352173 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.352154 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk"] Jun 02 20:10:24.440294 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.440266 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztxbg\" (UniqueName: \"kubernetes.io/projected/bd35c25e-7aea-4d00-a38b-8e162b884828-kube-api-access-ztxbg\") pod \"kube-auth-proxy-78d7545c57-vdmfk\" (UID: \"bd35c25e-7aea-4d00-a38b-8e162b884828\") " pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.440449 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.440312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd35c25e-7aea-4d00-a38b-8e162b884828-tmp\") pod \"kube-auth-proxy-78d7545c57-vdmfk\" (UID: \"bd35c25e-7aea-4d00-a38b-8e162b884828\") " pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.440449 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.440402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd35c25e-7aea-4d00-a38b-8e162b884828-tls-certs\") pod \"kube-auth-proxy-78d7545c57-vdmfk\" (UID: \"bd35c25e-7aea-4d00-a38b-8e162b884828\") " pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.541427 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.541390 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd35c25e-7aea-4d00-a38b-8e162b884828-tls-certs\") pod \"kube-auth-proxy-78d7545c57-vdmfk\" (UID: \"bd35c25e-7aea-4d00-a38b-8e162b884828\") " pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.541591 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.541472 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztxbg\" (UniqueName: \"kubernetes.io/projected/bd35c25e-7aea-4d00-a38b-8e162b884828-kube-api-access-ztxbg\") pod \"kube-auth-proxy-78d7545c57-vdmfk\" (UID: \"bd35c25e-7aea-4d00-a38b-8e162b884828\") " pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.541591 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.541520 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd35c25e-7aea-4d00-a38b-8e162b884828-tmp\") pod \"kube-auth-proxy-78d7545c57-vdmfk\" (UID: \"bd35c25e-7aea-4d00-a38b-8e162b884828\") " pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.543641 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.543607 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd35c25e-7aea-4d00-a38b-8e162b884828-tmp\") pod \"kube-auth-proxy-78d7545c57-vdmfk\" (UID: \"bd35c25e-7aea-4d00-a38b-8e162b884828\") " pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.543883 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.543860 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd35c25e-7aea-4d00-a38b-8e162b884828-tls-certs\") pod \"kube-auth-proxy-78d7545c57-vdmfk\" (UID: \"bd35c25e-7aea-4d00-a38b-8e162b884828\") " pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.551669 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.551648 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztxbg\" (UniqueName: \"kubernetes.io/projected/bd35c25e-7aea-4d00-a38b-8e162b884828-kube-api-access-ztxbg\") pod \"kube-auth-proxy-78d7545c57-vdmfk\" (UID: \"bd35c25e-7aea-4d00-a38b-8e162b884828\") " pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.649107 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.649033 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" Jun 02 20:10:24.778176 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.778143 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk"] Jun 02 20:10:24.781074 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:10:24.781047 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd35c25e_7aea_4d00_a38b_8e162b884828.slice/crio-5bd3dfc866d96bf649aaef28b6c6a2310c361055607b447354b782dd5cc6e936 WatchSource:0}: Error finding container 5bd3dfc866d96bf649aaef28b6c6a2310c361055607b447354b782dd5cc6e936: Status 404 returned error can't find the container with id 5bd3dfc866d96bf649aaef28b6c6a2310c361055607b447354b782dd5cc6e936 Jun 02 20:10:24.958465 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:24.958414 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" event={"ID":"bd35c25e-7aea-4d00-a38b-8e162b884828","Type":"ContainerStarted","Data":"5bd3dfc866d96bf649aaef28b6c6a2310c361055607b447354b782dd5cc6e936"} Jun 02 20:10:28.978328 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:28.978293 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" event={"ID":"bd35c25e-7aea-4d00-a38b-8e162b884828","Type":"ContainerStarted","Data":"ade52ef90d9860c79016d2befb127ea2273f96d507c58ab8225b9c704c293547"} Jun 02 20:10:28.997089 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:10:28.997042 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-78d7545c57-vdmfk" podStartSLOduration=1.369023726 podStartE2EDuration="4.99702772s" podCreationTimestamp="2026-06-02 20:10:24 +0000 UTC" firstStartedPulling="2026-06-02 20:10:24.782823359 +0000 UTC m=+500.980386033" lastFinishedPulling="2026-06-02 20:10:28.410827354 +0000 UTC m=+504.608390027" observedRunningTime="2026-06-02 20:10:28.995624644 +0000 UTC m=+505.193187341" watchObservedRunningTime="2026-06-02 20:10:28.99702772 +0000 UTC m=+505.194590434" Jun 02 20:12:04.335078 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:04.335049 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:12:04.335910 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:04.335892 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:12:12.181877 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.181836 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9"] Jun 02 20:12:12.184350 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.184329 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.186546 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.186523 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 02 20:12:12.186661 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.186556 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 20:12:12.186661 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.186638 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-92vdk\"" Jun 02 20:12:12.186661 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.186650 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 02 20:12:12.187347 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.187328 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 20:12:12.192672 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.192652 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9"] Jun 02 20:12:12.314014 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.313959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkzjq\" (UniqueName: \"kubernetes.io/projected/1c59d850-71c4-4b97-aa5f-6cec92bb74c1-kube-api-access-pkzjq\") pod \"kuadrant-console-plugin-6cb54b5c86-6mpz9\" (UID: \"1c59d850-71c4-4b97-aa5f-6cec92bb74c1\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.314172 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.314040 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c59d850-71c4-4b97-aa5f-6cec92bb74c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-6mpz9\" (UID: \"1c59d850-71c4-4b97-aa5f-6cec92bb74c1\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.314172 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.314067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1c59d850-71c4-4b97-aa5f-6cec92bb74c1-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-6mpz9\" (UID: \"1c59d850-71c4-4b97-aa5f-6cec92bb74c1\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.414493 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.414467 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pkzjq\" (UniqueName: \"kubernetes.io/projected/1c59d850-71c4-4b97-aa5f-6cec92bb74c1-kube-api-access-pkzjq\") pod \"kuadrant-console-plugin-6cb54b5c86-6mpz9\" (UID: \"1c59d850-71c4-4b97-aa5f-6cec92bb74c1\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.414673 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.414513 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c59d850-71c4-4b97-aa5f-6cec92bb74c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-6mpz9\" (UID: \"1c59d850-71c4-4b97-aa5f-6cec92bb74c1\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.414673 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.414539 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1c59d850-71c4-4b97-aa5f-6cec92bb74c1-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-6mpz9\" (UID: \"1c59d850-71c4-4b97-aa5f-6cec92bb74c1\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.415112 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.415091 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1c59d850-71c4-4b97-aa5f-6cec92bb74c1-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-6mpz9\" (UID: \"1c59d850-71c4-4b97-aa5f-6cec92bb74c1\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.417104 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.417081 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c59d850-71c4-4b97-aa5f-6cec92bb74c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-6mpz9\" (UID: \"1c59d850-71c4-4b97-aa5f-6cec92bb74c1\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.422016 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.421975 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkzjq\" (UniqueName: \"kubernetes.io/projected/1c59d850-71c4-4b97-aa5f-6cec92bb74c1-kube-api-access-pkzjq\") pod \"kuadrant-console-plugin-6cb54b5c86-6mpz9\" (UID: \"1c59d850-71c4-4b97-aa5f-6cec92bb74c1\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.494077 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.493976 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" Jun 02 20:12:12.618165 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:12.618135 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9"] Jun 02 20:12:12.621516 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:12:12.621487 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c59d850_71c4_4b97_aa5f_6cec92bb74c1.slice/crio-544082ef8e8ba22be9b6a2019fbea8ae1660c32423599bf0a3786f056db5d03a WatchSource:0}: Error finding container 544082ef8e8ba22be9b6a2019fbea8ae1660c32423599bf0a3786f056db5d03a: Status 404 returned error can't find the container with id 544082ef8e8ba22be9b6a2019fbea8ae1660c32423599bf0a3786f056db5d03a Jun 02 20:12:13.308148 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:13.308115 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" event={"ID":"1c59d850-71c4-4b97-aa5f-6cec92bb74c1","Type":"ContainerStarted","Data":"544082ef8e8ba22be9b6a2019fbea8ae1660c32423599bf0a3786f056db5d03a"} Jun 02 20:12:22.692029 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:22.691971 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8"] Jun 02 20:12:22.694451 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:22.694433 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" Jun 02 20:12:22.696736 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:22.696716 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mb766\"" Jun 02 20:12:22.705428 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:22.705403 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8"] Jun 02 20:12:22.807451 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:22.807361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwdlr\" (UniqueName: \"kubernetes.io/projected/117331b3-1a4d-4d03-adab-dd178113c2cd-kube-api-access-cwdlr\") pod \"limitador-operator-controller-manager-85c4996f8c-wsml8\" (UID: \"117331b3-1a4d-4d03-adab-dd178113c2cd\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" Jun 02 20:12:22.908376 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:22.908339 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cwdlr\" (UniqueName: \"kubernetes.io/projected/117331b3-1a4d-4d03-adab-dd178113c2cd-kube-api-access-cwdlr\") pod \"limitador-operator-controller-manager-85c4996f8c-wsml8\" (UID: \"117331b3-1a4d-4d03-adab-dd178113c2cd\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" Jun 02 20:12:22.927775 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:22.927747 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwdlr\" (UniqueName: \"kubernetes.io/projected/117331b3-1a4d-4d03-adab-dd178113c2cd-kube-api-access-cwdlr\") pod \"limitador-operator-controller-manager-85c4996f8c-wsml8\" (UID: \"117331b3-1a4d-4d03-adab-dd178113c2cd\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" Jun 02 20:12:23.005062 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:23.004969 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" Jun 02 20:12:23.143747 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:23.143720 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8"] Jun 02 20:12:23.146600 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:12:23.146569 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117331b3_1a4d_4d03_adab_dd178113c2cd.slice/crio-f3ae1a45f7864bfaedebdad07c47515911dece86114a6b5c97cf5acdb2d9197a WatchSource:0}: Error finding container f3ae1a45f7864bfaedebdad07c47515911dece86114a6b5c97cf5acdb2d9197a: Status 404 returned error can't find the container with id f3ae1a45f7864bfaedebdad07c47515911dece86114a6b5c97cf5acdb2d9197a Jun 02 20:12:23.350062 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:23.349947 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" event={"ID":"117331b3-1a4d-4d03-adab-dd178113c2cd","Type":"ContainerStarted","Data":"f3ae1a45f7864bfaedebdad07c47515911dece86114a6b5c97cf5acdb2d9197a"} Jun 02 20:12:39.407558 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:39.407520 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" event={"ID":"1c59d850-71c4-4b97-aa5f-6cec92bb74c1","Type":"ContainerStarted","Data":"69b3fc8b4fd5f4d409b91b107200eb6918f24504e691f9852722042d2f40f069"} Jun 02 20:12:39.425917 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:39.425865 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-6mpz9" podStartSLOduration=1.623398868 podStartE2EDuration="27.425851416s" podCreationTimestamp="2026-06-02 20:12:12 +0000 UTC" firstStartedPulling="2026-06-02 20:12:12.623152877 +0000 UTC m=+608.820715550" lastFinishedPulling="2026-06-02 20:12:38.425605424 +0000 UTC m=+634.623168098" observedRunningTime="2026-06-02 20:12:39.425129165 +0000 UTC m=+635.622691862" watchObservedRunningTime="2026-06-02 20:12:39.425851416 +0000 UTC m=+635.623414111" Jun 02 20:12:40.411466 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:40.411431 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" event={"ID":"117331b3-1a4d-4d03-adab-dd178113c2cd","Type":"ContainerStarted","Data":"2cf0e09dc387eb7f8ab89a65eb09f8e72cd4ae5dc497d4104b14350fbd64a0d3"} Jun 02 20:12:40.431969 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:40.431922 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" podStartSLOduration=1.955928766 podStartE2EDuration="18.431909884s" podCreationTimestamp="2026-06-02 20:12:22 +0000 UTC" firstStartedPulling="2026-06-02 20:12:23.14845712 +0000 UTC m=+619.346019793" lastFinishedPulling="2026-06-02 20:12:39.624438225 +0000 UTC m=+635.822000911" observedRunningTime="2026-06-02 20:12:40.430473692 +0000 UTC m=+636.628036393" watchObservedRunningTime="2026-06-02 20:12:40.431909884 +0000 UTC m=+636.629472579" Jun 02 20:12:41.414782 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:41.414752 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" Jun 02 20:12:52.420407 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:12:52.420375 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wsml8" Jun 02 20:13:07.349983 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:07.349948 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-wg9sm"] Jun 02 20:13:07.363146 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:07.363118 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" Jun 02 20:13:07.365971 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:07.365946 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5llhj\"" Jun 02 20:13:07.368465 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:07.368441 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-wg9sm"] Jun 02 20:13:07.477199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:07.477173 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngnfx\" (UniqueName: \"kubernetes.io/projected/8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b-kube-api-access-ngnfx\") pod \"authorino-f99f4b5cd-wg9sm\" (UID: \"8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b\") " pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" Jun 02 20:13:07.578185 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:07.578156 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ngnfx\" (UniqueName: \"kubernetes.io/projected/8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b-kube-api-access-ngnfx\") pod \"authorino-f99f4b5cd-wg9sm\" (UID: \"8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b\") " pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" Jun 02 20:13:07.586410 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:07.586390 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngnfx\" (UniqueName: \"kubernetes.io/projected/8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b-kube-api-access-ngnfx\") pod \"authorino-f99f4b5cd-wg9sm\" (UID: \"8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b\") " pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" Jun 02 20:13:07.673036 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:07.672980 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" Jun 02 20:13:07.798065 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:07.798033 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-wg9sm"] Jun 02 20:13:07.801285 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:13:07.801257 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8424ead1_b3a1_43dc_8f2b_0c18aaa59a1b.slice/crio-a44209a4513386b6a14ed5e9b31fdac4b9b21f28931b00fe8913bb3faba4b80e WatchSource:0}: Error finding container a44209a4513386b6a14ed5e9b31fdac4b9b21f28931b00fe8913bb3faba4b80e: Status 404 returned error can't find the container with id a44209a4513386b6a14ed5e9b31fdac4b9b21f28931b00fe8913bb3faba4b80e Jun 02 20:13:08.500336 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:08.500293 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" event={"ID":"8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b","Type":"ContainerStarted","Data":"a44209a4513386b6a14ed5e9b31fdac4b9b21f28931b00fe8913bb3faba4b80e"} Jun 02 20:13:11.511914 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:11.511877 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" event={"ID":"8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b","Type":"ContainerStarted","Data":"47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d"} Jun 02 20:13:11.529032 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:11.528973 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" podStartSLOduration=1.337571641 podStartE2EDuration="4.528960524s" podCreationTimestamp="2026-06-02 20:13:07 +0000 UTC" firstStartedPulling="2026-06-02 20:13:07.802534565 +0000 UTC m=+664.000097238" lastFinishedPulling="2026-06-02 20:13:10.993923431 +0000 UTC m=+667.191486121" observedRunningTime="2026-06-02 20:13:11.527210002 +0000 UTC m=+667.724772698" watchObservedRunningTime="2026-06-02 20:13:11.528960524 +0000 UTC m=+667.726523244" Jun 02 20:13:12.185789 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:12.185759 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-wg9sm"] Jun 02 20:13:13.518476 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:13.518437 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" podUID="8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b" containerName="authorino" containerID="cri-o://47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d" gracePeriod=30 Jun 02 20:13:13.764651 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:13.764631 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" Jun 02 20:13:13.835500 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:13.835433 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngnfx\" (UniqueName: \"kubernetes.io/projected/8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b-kube-api-access-ngnfx\") pod \"8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b\" (UID: \"8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b\") " Jun 02 20:13:13.837562 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:13.837538 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b-kube-api-access-ngnfx" (OuterVolumeSpecName: "kube-api-access-ngnfx") pod "8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b" (UID: "8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b"). InnerVolumeSpecName "kube-api-access-ngnfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:13:13.936123 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:13.936089 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngnfx\" (UniqueName: \"kubernetes.io/projected/8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b-kube-api-access-ngnfx\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:13:14.522634 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:14.522596 2577 generic.go:358] "Generic (PLEG): container finished" podID="8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b" containerID="47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d" exitCode=0 Jun 02 20:13:14.523060 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:14.522649 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" Jun 02 20:13:14.523060 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:14.522680 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" event={"ID":"8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b","Type":"ContainerDied","Data":"47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d"} Jun 02 20:13:14.523060 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:14.522716 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-wg9sm" event={"ID":"8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b","Type":"ContainerDied","Data":"a44209a4513386b6a14ed5e9b31fdac4b9b21f28931b00fe8913bb3faba4b80e"} Jun 02 20:13:14.523060 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:14.522732 2577 scope.go:117] "RemoveContainer" containerID="47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d" Jun 02 20:13:14.530610 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:14.530589 2577 scope.go:117] "RemoveContainer" containerID="47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d" Jun 02 20:13:14.530849 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:13:14.530831 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d\": container with ID starting with 47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d not found: ID does not exist" containerID="47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d" Jun 02 20:13:14.530894 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:14.530856 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d"} err="failed to get container status \"47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d\": rpc error: code = NotFound desc = could not find container \"47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d\": container with ID starting with 47f18ace5281c3b70af3c7a911fade4b0772bff31afacb705fdb64d54229a75d not found: ID does not exist" Jun 02 20:13:14.539041 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:14.539017 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-wg9sm"] Jun 02 20:13:14.542339 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:14.542318 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-wg9sm"] Jun 02 20:13:16.441304 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:13:16.441271 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b" path="/var/lib/kubelet/pods/8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b/volumes" Jun 02 20:14:02.579881 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.579801 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 20:14:02.580318 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.580164 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b" containerName="authorino" Jun 02 20:14:02.580318 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.580177 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b" containerName="authorino" Jun 02 20:14:02.580318 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.580235 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8424ead1-b3a1-43dc-8f2b-0c18aaa59a1b" containerName="authorino" Jun 02 20:14:02.582499 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.582480 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 20:14:02.585390 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.585150 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 20:14:02.585390 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.585238 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 20:14:02.585390 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.585317 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 20:14:02.588009 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.585845 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-fvlcw\"" Jun 02 20:14:02.593071 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.593047 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 20:14:02.736522 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.736488 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-624km\" (UniqueName: \"kubernetes.io/projected/dae6b47e-0c7c-48a6-a85a-ea32cca6bf74-kube-api-access-624km\") pod \"maas-keycloak-0\" (UID: \"dae6b47e-0c7c-48a6-a85a-ea32cca6bf74\") " pod="keycloak-system/maas-keycloak-0" Jun 02 20:14:02.837957 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.837856 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-624km\" (UniqueName: \"kubernetes.io/projected/dae6b47e-0c7c-48a6-a85a-ea32cca6bf74-kube-api-access-624km\") pod \"maas-keycloak-0\" (UID: \"dae6b47e-0c7c-48a6-a85a-ea32cca6bf74\") " pod="keycloak-system/maas-keycloak-0" Jun 02 20:14:02.847435 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.847401 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-624km\" (UniqueName: \"kubernetes.io/projected/dae6b47e-0c7c-48a6-a85a-ea32cca6bf74-kube-api-access-624km\") pod \"maas-keycloak-0\" (UID: \"dae6b47e-0c7c-48a6-a85a-ea32cca6bf74\") " pod="keycloak-system/maas-keycloak-0" Jun 02 20:14:02.897710 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:02.897674 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 20:14:03.028677 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:03.028652 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 20:14:03.031403 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:14:03.031380 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddae6b47e_0c7c_48a6_a85a_ea32cca6bf74.slice/crio-f76fc16c56dd448d087725c84532bf6be2c87ea7a6b968b97ba1882899be4658 WatchSource:0}: Error finding container f76fc16c56dd448d087725c84532bf6be2c87ea7a6b968b97ba1882899be4658: Status 404 returned error can't find the container with id f76fc16c56dd448d087725c84532bf6be2c87ea7a6b968b97ba1882899be4658 Jun 02 20:14:03.679775 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:03.679734 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"dae6b47e-0c7c-48a6-a85a-ea32cca6bf74","Type":"ContainerStarted","Data":"f76fc16c56dd448d087725c84532bf6be2c87ea7a6b968b97ba1882899be4658"} Jun 02 20:14:07.797588 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:07.797561 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 20:14:08.702286 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:08.702244 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"dae6b47e-0c7c-48a6-a85a-ea32cca6bf74","Type":"ContainerStarted","Data":"5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21"} Jun 02 20:14:08.721565 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:08.721511 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.959512021 podStartE2EDuration="6.721496597s" podCreationTimestamp="2026-06-02 20:14:02 +0000 UTC" firstStartedPulling="2026-06-02 20:14:03.033048521 +0000 UTC m=+719.230611195" lastFinishedPulling="2026-06-02 20:14:07.795033081 +0000 UTC m=+723.992595771" observedRunningTime="2026-06-02 20:14:08.719833863 +0000 UTC m=+724.917396560" watchObservedRunningTime="2026-06-02 20:14:08.721496597 +0000 UTC m=+724.919059292" Jun 02 20:14:08.898498 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:08.898457 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 20:14:08.900053 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:08.900006 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:09.898833 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:09.898780 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:10.899008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:10.898943 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:11.898464 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:11.898414 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:12.899482 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:12.898787 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 20:14:12.899482 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:12.899123 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:13.898736 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:13.898681 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:14.899847 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:14.899794 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:15.898187 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:15.898136 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:16.898550 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:16.898502 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:17.899177 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:17.899130 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:18.898798 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:18.898749 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:19.898674 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:19.898624 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.30:9000/health/started\": dial tcp 10.133.0.30:9000: connect: connection refused" Jun 02 20:14:21.013812 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:21.013773 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 20:14:21.033834 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:21.033781 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 20:14:31.020173 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:31.020137 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 20:14:36.783165 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.783126 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f8c8c894f-5vdqn"] Jun 02 20:14:36.789208 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.789190 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" Jun 02 20:14:36.791815 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.791793 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h65xp\"" Jun 02 20:14:36.792260 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.792243 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 20:14:36.794773 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.794754 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f8c8c894f-5vdqn"] Jun 02 20:14:36.859202 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.859173 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8txgx\" (UniqueName: \"kubernetes.io/projected/c3298e42-278a-4d42-81a7-f81e918c8e46-kube-api-access-8txgx\") pod \"maas-controller-5f8c8c894f-5vdqn\" (UID: \"c3298e42-278a-4d42-81a7-f81e918c8e46\") " pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" Jun 02 20:14:36.959140 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.959100 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bb676c484-h5zrv"] Jun 02 20:14:36.959913 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.959891 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8txgx\" (UniqueName: \"kubernetes.io/projected/c3298e42-278a-4d42-81a7-f81e918c8e46-kube-api-access-8txgx\") pod \"maas-controller-5f8c8c894f-5vdqn\" (UID: \"c3298e42-278a-4d42-81a7-f81e918c8e46\") " pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" Jun 02 20:14:36.961510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.961494 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb676c484-h5zrv" Jun 02 20:14:36.968415 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.968382 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8txgx\" (UniqueName: \"kubernetes.io/projected/c3298e42-278a-4d42-81a7-f81e918c8e46-kube-api-access-8txgx\") pod \"maas-controller-5f8c8c894f-5vdqn\" (UID: \"c3298e42-278a-4d42-81a7-f81e918c8e46\") " pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" Jun 02 20:14:36.971065 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:36.971041 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb676c484-h5zrv"] Jun 02 20:14:37.060956 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.060871 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8z2h\" (UniqueName: \"kubernetes.io/projected/6617307f-6a73-4eab-9d1a-68a1caca4c38-kube-api-access-t8z2h\") pod \"maas-controller-7bb676c484-h5zrv\" (UID: \"6617307f-6a73-4eab-9d1a-68a1caca4c38\") " pod="opendatahub/maas-controller-7bb676c484-h5zrv" Jun 02 20:14:37.099743 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.099697 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" Jun 02 20:14:37.161646 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.161610 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t8z2h\" (UniqueName: \"kubernetes.io/projected/6617307f-6a73-4eab-9d1a-68a1caca4c38-kube-api-access-t8z2h\") pod \"maas-controller-7bb676c484-h5zrv\" (UID: \"6617307f-6a73-4eab-9d1a-68a1caca4c38\") " pod="opendatahub/maas-controller-7bb676c484-h5zrv" Jun 02 20:14:37.171615 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.171576 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8z2h\" (UniqueName: \"kubernetes.io/projected/6617307f-6a73-4eab-9d1a-68a1caca4c38-kube-api-access-t8z2h\") pod \"maas-controller-7bb676c484-h5zrv\" (UID: \"6617307f-6a73-4eab-9d1a-68a1caca4c38\") " pod="opendatahub/maas-controller-7bb676c484-h5zrv" Jun 02 20:14:37.229877 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.229733 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f8c8c894f-5vdqn"] Jun 02 20:14:37.232523 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:14:37.232489 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3298e42_278a_4d42_81a7_f81e918c8e46.slice/crio-cf40190679b28e7ffab384192252cb0aeeae3c25b38b2a8398541e87742ea655 WatchSource:0}: Error finding container cf40190679b28e7ffab384192252cb0aeeae3c25b38b2a8398541e87742ea655: Status 404 returned error can't find the container with id cf40190679b28e7ffab384192252cb0aeeae3c25b38b2a8398541e87742ea655 Jun 02 20:14:37.233761 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.233743 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 20:14:37.282250 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.282223 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb676c484-h5zrv" Jun 02 20:14:37.408967 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.408943 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb676c484-h5zrv"] Jun 02 20:14:37.411453 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:14:37.411424 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6617307f_6a73_4eab_9d1a_68a1caca4c38.slice/crio-951f61cb7f4ba816f554d275e7859a8d8d9b4a625955660b830b8057f82ca6e3 WatchSource:0}: Error finding container 951f61cb7f4ba816f554d275e7859a8d8d9b4a625955660b830b8057f82ca6e3: Status 404 returned error can't find the container with id 951f61cb7f4ba816f554d275e7859a8d8d9b4a625955660b830b8057f82ca6e3 Jun 02 20:14:37.813927 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.813887 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb676c484-h5zrv" event={"ID":"6617307f-6a73-4eab-9d1a-68a1caca4c38","Type":"ContainerStarted","Data":"951f61cb7f4ba816f554d275e7859a8d8d9b4a625955660b830b8057f82ca6e3"} Jun 02 20:14:37.815019 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:37.814971 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" event={"ID":"c3298e42-278a-4d42-81a7-f81e918c8e46","Type":"ContainerStarted","Data":"cf40190679b28e7ffab384192252cb0aeeae3c25b38b2a8398541e87742ea655"} Jun 02 20:14:40.827284 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:40.827245 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" event={"ID":"c3298e42-278a-4d42-81a7-f81e918c8e46","Type":"ContainerStarted","Data":"eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04"} Jun 02 20:14:40.827723 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:40.827293 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" Jun 02 20:14:40.828599 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:40.828573 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb676c484-h5zrv" event={"ID":"6617307f-6a73-4eab-9d1a-68a1caca4c38","Type":"ContainerStarted","Data":"9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6"} Jun 02 20:14:40.828711 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:40.828684 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bb676c484-h5zrv" Jun 02 20:14:40.859382 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:40.858509 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" podStartSLOduration=2.1602166 podStartE2EDuration="4.858493403s" podCreationTimestamp="2026-06-02 20:14:36 +0000 UTC" firstStartedPulling="2026-06-02 20:14:37.233871243 +0000 UTC m=+753.431433915" lastFinishedPulling="2026-06-02 20:14:39.932148045 +0000 UTC m=+756.129710718" observedRunningTime="2026-06-02 20:14:40.853511133 +0000 UTC m=+757.051073829" watchObservedRunningTime="2026-06-02 20:14:40.858493403 +0000 UTC m=+757.056056100" Jun 02 20:14:51.837182 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:51.837145 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" Jun 02 20:14:51.837665 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:51.837448 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bb676c484-h5zrv" Jun 02 20:14:51.855400 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:51.855351 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7bb676c484-h5zrv" podStartSLOduration=13.345217472 podStartE2EDuration="15.855330778s" podCreationTimestamp="2026-06-02 20:14:36 +0000 UTC" firstStartedPulling="2026-06-02 20:14:37.412818934 +0000 UTC m=+753.610381606" lastFinishedPulling="2026-06-02 20:14:39.922932237 +0000 UTC m=+756.120494912" observedRunningTime="2026-06-02 20:14:40.88644156 +0000 UTC m=+757.084004265" watchObservedRunningTime="2026-06-02 20:14:51.855330778 +0000 UTC m=+768.052893491" Jun 02 20:14:51.894176 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:51.894137 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f8c8c894f-5vdqn"] Jun 02 20:14:51.894409 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:51.894365 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" podUID="c3298e42-278a-4d42-81a7-f81e918c8e46" containerName="manager" containerID="cri-o://eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04" gracePeriod=30 Jun 02 20:14:52.138727 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.138702 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" Jun 02 20:14:52.293641 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.293609 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8txgx\" (UniqueName: \"kubernetes.io/projected/c3298e42-278a-4d42-81a7-f81e918c8e46-kube-api-access-8txgx\") pod \"c3298e42-278a-4d42-81a7-f81e918c8e46\" (UID: \"c3298e42-278a-4d42-81a7-f81e918c8e46\") " Jun 02 20:14:52.295798 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.295774 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3298e42-278a-4d42-81a7-f81e918c8e46-kube-api-access-8txgx" (OuterVolumeSpecName: "kube-api-access-8txgx") pod "c3298e42-278a-4d42-81a7-f81e918c8e46" (UID: "c3298e42-278a-4d42-81a7-f81e918c8e46"). InnerVolumeSpecName "kube-api-access-8txgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:14:52.394978 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.394908 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8txgx\" (UniqueName: \"kubernetes.io/projected/c3298e42-278a-4d42-81a7-f81e918c8e46-kube-api-access-8txgx\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:14:52.868121 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.868087 2577 generic.go:358] "Generic (PLEG): container finished" podID="c3298e42-278a-4d42-81a7-f81e918c8e46" containerID="eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04" exitCode=0 Jun 02 20:14:52.868542 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.868142 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" event={"ID":"c3298e42-278a-4d42-81a7-f81e918c8e46","Type":"ContainerDied","Data":"eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04"} Jun 02 20:14:52.868542 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.868154 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" Jun 02 20:14:52.868542 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.868166 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f8c8c894f-5vdqn" event={"ID":"c3298e42-278a-4d42-81a7-f81e918c8e46","Type":"ContainerDied","Data":"cf40190679b28e7ffab384192252cb0aeeae3c25b38b2a8398541e87742ea655"} Jun 02 20:14:52.868542 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.868181 2577 scope.go:117] "RemoveContainer" containerID="eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04" Jun 02 20:14:52.876060 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.876044 2577 scope.go:117] "RemoveContainer" containerID="eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04" Jun 02 20:14:52.876325 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:14:52.876297 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04\": container with ID starting with eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04 not found: ID does not exist" containerID="eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04" Jun 02 20:14:52.876401 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.876326 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04"} err="failed to get container status \"eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04\": rpc error: code = NotFound desc = could not find container \"eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04\": container with ID starting with eb23f0f699b57aae91c7d6790b60304f76d483736572f27f06294c70ae80fb04 not found: ID does not exist" Jun 02 20:14:52.884911 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.884886 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f8c8c894f-5vdqn"] Jun 02 20:14:52.887208 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:52.887189 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f8c8c894f-5vdqn"] Jun 02 20:14:54.440568 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:14:54.440532 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3298e42-278a-4d42-81a7-f81e918c8e46" path="/var/lib/kubelet/pods/c3298e42-278a-4d42-81a7-f81e918c8e46/volumes" Jun 02 20:15:11.220484 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.220446 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-df756cc49-qjn2v"] Jun 02 20:15:11.220929 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.220769 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3298e42-278a-4d42-81a7-f81e918c8e46" containerName="manager" Jun 02 20:15:11.220929 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.220780 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3298e42-278a-4d42-81a7-f81e918c8e46" containerName="manager" Jun 02 20:15:11.220929 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.220847 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="c3298e42-278a-4d42-81a7-f81e918c8e46" containerName="manager" Jun 02 20:15:11.225186 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.225166 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:11.227823 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.227807 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-b2dh4\"" Jun 02 20:15:11.228107 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.228090 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 02 20:15:11.233858 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.233835 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-df756cc49-qjn2v"] Jun 02 20:15:11.253469 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.253441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f3b48fa-0b04-4322-9b42-84c1159cce02-maas-api-tls\") pod \"maas-api-df756cc49-qjn2v\" (UID: \"5f3b48fa-0b04-4322-9b42-84c1159cce02\") " pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:11.253564 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.253479 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfvg5\" (UniqueName: \"kubernetes.io/projected/5f3b48fa-0b04-4322-9b42-84c1159cce02-kube-api-access-gfvg5\") pod \"maas-api-df756cc49-qjn2v\" (UID: \"5f3b48fa-0b04-4322-9b42-84c1159cce02\") " pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:11.354485 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.354455 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gfvg5\" (UniqueName: \"kubernetes.io/projected/5f3b48fa-0b04-4322-9b42-84c1159cce02-kube-api-access-gfvg5\") pod \"maas-api-df756cc49-qjn2v\" (UID: \"5f3b48fa-0b04-4322-9b42-84c1159cce02\") " pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:11.354605 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.354527 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f3b48fa-0b04-4322-9b42-84c1159cce02-maas-api-tls\") pod \"maas-api-df756cc49-qjn2v\" (UID: \"5f3b48fa-0b04-4322-9b42-84c1159cce02\") " pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:11.354647 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:15:11.354607 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 02 20:15:11.354680 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:15:11.354664 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f3b48fa-0b04-4322-9b42-84c1159cce02-maas-api-tls podName:5f3b48fa-0b04-4322-9b42-84c1159cce02 nodeName:}" failed. No retries permitted until 2026-06-02 20:15:11.854649404 +0000 UTC m=+788.052212077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f3b48fa-0b04-4322-9b42-84c1159cce02-maas-api-tls") pod "maas-api-df756cc49-qjn2v" (UID: "5f3b48fa-0b04-4322-9b42-84c1159cce02") : secret "maas-api-serving-cert" not found Jun 02 20:15:11.365921 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.365901 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfvg5\" (UniqueName: \"kubernetes.io/projected/5f3b48fa-0b04-4322-9b42-84c1159cce02-kube-api-access-gfvg5\") pod \"maas-api-df756cc49-qjn2v\" (UID: \"5f3b48fa-0b04-4322-9b42-84c1159cce02\") " pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:11.605075 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.604987 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-trk6s"] Jun 02 20:15:11.608331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.608315 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" Jun 02 20:15:11.610845 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.610821 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 02 20:15:11.611087 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.611071 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-45tpv\"" Jun 02 20:15:11.618638 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.618611 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-trk6s"] Jun 02 20:15:11.658340 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.658312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmxc4\" (UniqueName: \"kubernetes.io/projected/3f320db7-015f-4b20-990a-5918ba804edb-kube-api-access-dmxc4\") pod \"payload-processing-58b6c8fdc7-trk6s\" (UID: \"3f320db7-015f-4b20-990a-5918ba804edb\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" Jun 02 20:15:11.759515 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.759479 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dmxc4\" (UniqueName: \"kubernetes.io/projected/3f320db7-015f-4b20-990a-5918ba804edb-kube-api-access-dmxc4\") pod \"payload-processing-58b6c8fdc7-trk6s\" (UID: \"3f320db7-015f-4b20-990a-5918ba804edb\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" Jun 02 20:15:11.780415 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.780389 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmxc4\" (UniqueName: \"kubernetes.io/projected/3f320db7-015f-4b20-990a-5918ba804edb-kube-api-access-dmxc4\") pod \"payload-processing-58b6c8fdc7-trk6s\" (UID: \"3f320db7-015f-4b20-990a-5918ba804edb\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" Jun 02 20:15:11.860337 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.860263 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f3b48fa-0b04-4322-9b42-84c1159cce02-maas-api-tls\") pod \"maas-api-df756cc49-qjn2v\" (UID: \"5f3b48fa-0b04-4322-9b42-84c1159cce02\") " pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:11.862645 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.862615 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f3b48fa-0b04-4322-9b42-84c1159cce02-maas-api-tls\") pod \"maas-api-df756cc49-qjn2v\" (UID: \"5f3b48fa-0b04-4322-9b42-84c1159cce02\") " pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:11.920428 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:11.920398 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" Jun 02 20:15:12.063290 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:12.063263 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-trk6s"] Jun 02 20:15:12.066077 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:15:12.066051 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f320db7_015f_4b20_990a_5918ba804edb.slice/crio-de8c42584f11b41e709f8be99193db1c0efd07cf7538f502b6db6399ca088faa WatchSource:0}: Error finding container de8c42584f11b41e709f8be99193db1c0efd07cf7538f502b6db6399ca088faa: Status 404 returned error can't find the container with id de8c42584f11b41e709f8be99193db1c0efd07cf7538f502b6db6399ca088faa Jun 02 20:15:12.136641 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:12.136571 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:12.270457 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:12.270426 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-df756cc49-qjn2v"] Jun 02 20:15:12.275551 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:15:12.275523 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f3b48fa_0b04_4322_9b42_84c1159cce02.slice/crio-71ec2e864386c81bc9f6509bfd605152369c4e26539404a43c2bd242ae3c9d24 WatchSource:0}: Error finding container 71ec2e864386c81bc9f6509bfd605152369c4e26539404a43c2bd242ae3c9d24: Status 404 returned error can't find the container with id 71ec2e864386c81bc9f6509bfd605152369c4e26539404a43c2bd242ae3c9d24 Jun 02 20:15:12.939767 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:12.939729 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" event={"ID":"3f320db7-015f-4b20-990a-5918ba804edb","Type":"ContainerStarted","Data":"de8c42584f11b41e709f8be99193db1c0efd07cf7538f502b6db6399ca088faa"} Jun 02 20:15:12.941786 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:12.941753 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-df756cc49-qjn2v" event={"ID":"5f3b48fa-0b04-4322-9b42-84c1159cce02","Type":"ContainerStarted","Data":"71ec2e864386c81bc9f6509bfd605152369c4e26539404a43c2bd242ae3c9d24"} Jun 02 20:15:14.951446 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:14.951405 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" event={"ID":"3f320db7-015f-4b20-990a-5918ba804edb","Type":"ContainerStarted","Data":"327c9fe6dc80fb8d1f607fac6549cb9ae29154462a6bc56f69890cebc59149fc"} Jun 02 20:15:14.952043 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:14.951495 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" Jun 02 20:15:14.953049 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:14.953023 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-df756cc49-qjn2v" event={"ID":"5f3b48fa-0b04-4322-9b42-84c1159cce02","Type":"ContainerStarted","Data":"df42d57dfe8f61f191180a82b32ac4e580fe5344014d6ea927f72ddf36d8b9a2"} Jun 02 20:15:14.953165 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:14.953131 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:14.973928 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:14.973887 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" podStartSLOduration=1.303603203 podStartE2EDuration="3.97387537s" podCreationTimestamp="2026-06-02 20:15:11 +0000 UTC" firstStartedPulling="2026-06-02 20:15:12.06788921 +0000 UTC m=+788.265451882" lastFinishedPulling="2026-06-02 20:15:14.738161375 +0000 UTC m=+790.935724049" observedRunningTime="2026-06-02 20:15:14.972150691 +0000 UTC m=+791.169713399" watchObservedRunningTime="2026-06-02 20:15:14.97387537 +0000 UTC m=+791.171438065" Jun 02 20:15:14.992817 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:14.992745 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-df756cc49-qjn2v" podStartSLOduration=1.532905301 podStartE2EDuration="3.99273382s" podCreationTimestamp="2026-06-02 20:15:11 +0000 UTC" firstStartedPulling="2026-06-02 20:15:12.276948579 +0000 UTC m=+788.474511252" lastFinishedPulling="2026-06-02 20:15:14.736777095 +0000 UTC m=+790.934339771" observedRunningTime="2026-06-02 20:15:14.991519138 +0000 UTC m=+791.189081834" watchObservedRunningTime="2026-06-02 20:15:14.99273382 +0000 UTC m=+791.190296514" Jun 02 20:15:20.962654 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:20.962624 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-df756cc49-qjn2v" Jun 02 20:15:22.863449 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:22.863364 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 20:15:22.863985 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:22.863669 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" containerID="cri-o://5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21" gracePeriod=30 Jun 02 20:15:24.410006 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.409976 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:24.466463 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.466435 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-624km\" (UniqueName: \"kubernetes.io/projected/dae6b47e-0c7c-48a6-a85a-ea32cca6bf74-kube-api-access-624km\") pod \"dae6b47e-0c7c-48a6-a85a-ea32cca6bf74\" (UID: \"dae6b47e-0c7c-48a6-a85a-ea32cca6bf74\") " Jun 02 20:15:24.468610 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.468584 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dae6b47e-0c7c-48a6-a85a-ea32cca6bf74-kube-api-access-624km" (OuterVolumeSpecName: "kube-api-access-624km") pod "dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" (UID: "dae6b47e-0c7c-48a6-a85a-ea32cca6bf74"). InnerVolumeSpecName "kube-api-access-624km". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:15:24.567563 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.567490 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-624km\" (UniqueName: \"kubernetes.io/projected/dae6b47e-0c7c-48a6-a85a-ea32cca6bf74-kube-api-access-624km\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:15:24.988332 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.988292 2577 generic.go:358] "Generic (PLEG): container finished" podID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerID="5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21" exitCode=143 Jun 02 20:15:24.988524 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.988356 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:24.988524 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.988382 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"dae6b47e-0c7c-48a6-a85a-ea32cca6bf74","Type":"ContainerDied","Data":"5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21"} Jun 02 20:15:24.988524 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.988426 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"dae6b47e-0c7c-48a6-a85a-ea32cca6bf74","Type":"ContainerDied","Data":"f76fc16c56dd448d087725c84532bf6be2c87ea7a6b968b97ba1882899be4658"} Jun 02 20:15:24.988524 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.988446 2577 scope.go:117] "RemoveContainer" containerID="5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21" Jun 02 20:15:24.997829 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.997811 2577 scope.go:117] "RemoveContainer" containerID="5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21" Jun 02 20:15:24.998069 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:15:24.998051 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21\": container with ID starting with 5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21 not found: ID does not exist" containerID="5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21" Jun 02 20:15:24.998118 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:24.998078 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21"} err="failed to get container status \"5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21\": rpc error: code = NotFound desc = could not find container \"5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21\": container with ID starting with 5935f0623e18454276c06532964190c249749e6231a06ffa8b58f55b493fab21 not found: ID does not exist" Jun 02 20:15:25.012390 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.012363 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 20:15:25.016072 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.016047 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 20:15:25.049163 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.049139 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 20:15:25.049498 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.049480 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" Jun 02 20:15:25.049498 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.049496 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" Jun 02 20:15:25.049601 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.049585 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" containerName="keycloak" Jun 02 20:15:25.052196 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.052180 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:25.054704 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.054682 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 20:15:25.054804 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.054702 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 20:15:25.054804 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.054771 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 20:15:25.054804 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.054793 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 02 20:15:25.055073 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.055059 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-fvlcw\"" Jun 02 20:15:25.061498 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.061477 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 20:15:25.173331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.173296 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wmmq\" (UniqueName: \"kubernetes.io/projected/23bfad07-b6ce-4d92-88e8-d24c7f645137-kube-api-access-2wmmq\") pod \"maas-keycloak-0\" (UID: \"23bfad07-b6ce-4d92-88e8-d24c7f645137\") " pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:25.173505 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.173397 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/23bfad07-b6ce-4d92-88e8-d24c7f645137-test-realms\") pod \"maas-keycloak-0\" (UID: \"23bfad07-b6ce-4d92-88e8-d24c7f645137\") " pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:25.274331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.274230 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/23bfad07-b6ce-4d92-88e8-d24c7f645137-test-realms\") pod \"maas-keycloak-0\" (UID: \"23bfad07-b6ce-4d92-88e8-d24c7f645137\") " pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:25.274331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.274314 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2wmmq\" (UniqueName: \"kubernetes.io/projected/23bfad07-b6ce-4d92-88e8-d24c7f645137-kube-api-access-2wmmq\") pod \"maas-keycloak-0\" (UID: \"23bfad07-b6ce-4d92-88e8-d24c7f645137\") " pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:25.274881 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.274859 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/23bfad07-b6ce-4d92-88e8-d24c7f645137-test-realms\") pod \"maas-keycloak-0\" (UID: \"23bfad07-b6ce-4d92-88e8-d24c7f645137\") " pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:25.283348 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.283317 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wmmq\" (UniqueName: \"kubernetes.io/projected/23bfad07-b6ce-4d92-88e8-d24c7f645137-kube-api-access-2wmmq\") pod \"maas-keycloak-0\" (UID: \"23bfad07-b6ce-4d92-88e8-d24c7f645137\") " pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:25.361775 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.361735 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:25.493637 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.493601 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 20:15:25.497006 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:15:25.496965 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23bfad07_b6ce_4d92_88e8_d24c7f645137.slice/crio-e7df47e924779c355a6e72557d76eede037cbc5fa9bb4397bfe35ca53b2dcf72 WatchSource:0}: Error finding container e7df47e924779c355a6e72557d76eede037cbc5fa9bb4397bfe35ca53b2dcf72: Status 404 returned error can't find the container with id e7df47e924779c355a6e72557d76eede037cbc5fa9bb4397bfe35ca53b2dcf72 Jun 02 20:15:25.960183 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.960149 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-trk6s" Jun 02 20:15:26.000117 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.995287 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"23bfad07-b6ce-4d92-88e8-d24c7f645137","Type":"ContainerStarted","Data":"4d3387681aa3fc74901e8a57f91b3202e72340d6b8a54cff2133db8b5c76fceb"} Jun 02 20:15:26.000117 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:25.995327 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"23bfad07-b6ce-4d92-88e8-d24c7f645137","Type":"ContainerStarted","Data":"e7df47e924779c355a6e72557d76eede037cbc5fa9bb4397bfe35ca53b2dcf72"} Jun 02 20:15:26.021863 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:26.021802 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.675179941 podStartE2EDuration="1.021783388s" podCreationTimestamp="2026-06-02 20:15:25 +0000 UTC" firstStartedPulling="2026-06-02 20:15:25.498816602 +0000 UTC m=+801.696379276" lastFinishedPulling="2026-06-02 20:15:25.845419838 +0000 UTC m=+802.042982723" observedRunningTime="2026-06-02 20:15:26.018743689 +0000 UTC m=+802.216306385" watchObservedRunningTime="2026-06-02 20:15:26.021783388 +0000 UTC m=+802.219346085" Jun 02 20:15:26.362590 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:26.362549 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:26.364544 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:26.364502 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:26.442719 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:26.442678 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dae6b47e-0c7c-48a6-a85a-ea32cca6bf74" path="/var/lib/kubelet/pods/dae6b47e-0c7c-48a6-a85a-ea32cca6bf74/volumes" Jun 02 20:15:27.363075 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:27.363027 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:28.362556 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:28.362501 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:29.362836 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:29.362791 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:30.362939 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:30.362880 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:31.363166 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:31.363115 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:32.362868 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:32.362818 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:33.362586 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:33.362539 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:34.362467 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:34.362415 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:35.362772 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:35.362726 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:35.363295 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:35.363040 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:36.362690 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:36.362627 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:37.362259 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:37.362217 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:38.362236 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:38.362186 2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 02 20:15:39.473245 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:39.473196 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 20:15:39.492114 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:39.492047 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="23bfad07-b6ce-4d92-88e8-d24c7f645137" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 20:15:49.480329 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:15:49.480292 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 20:16:28.508804 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.508767 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg"] Jun 02 20:16:28.512316 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.512299 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.515480 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.515453 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 20:16:28.515580 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.515455 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 02 20:16:28.515580 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.515455 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 20:16:28.515580 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.515541 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-glkvx\"" Jun 02 20:16:28.524954 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.524932 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg"] Jun 02 20:16:28.648594 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.648559 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx84r\" (UniqueName: \"kubernetes.io/projected/865a7984-70fd-4b47-9318-97b13fe46c62-kube-api-access-tx84r\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.648777 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.648608 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.648777 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.648669 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.648777 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.648737 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.648777 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.648761 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865a7984-70fd-4b47-9318-97b13fe46c62-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.648960 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.648822 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.749661 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.749629 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.749812 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.749681 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.749812 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.749723 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.749812 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.749745 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865a7984-70fd-4b47-9318-97b13fe46c62-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.749812 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.749785 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.750040 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.749837 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tx84r\" (UniqueName: \"kubernetes.io/projected/865a7984-70fd-4b47-9318-97b13fe46c62-kube-api-access-tx84r\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.750130 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.750111 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.750184 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.750161 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.750233 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.750192 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.751911 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.751888 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865a7984-70fd-4b47-9318-97b13fe46c62-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.752199 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.752182 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865a7984-70fd-4b47-9318-97b13fe46c62-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.760239 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.760188 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx84r\" (UniqueName: \"kubernetes.io/projected/865a7984-70fd-4b47-9318-97b13fe46c62-kube-api-access-tx84r\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg\" (UID: \"865a7984-70fd-4b47-9318-97b13fe46c62\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.822827 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.822804 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:28.953671 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:28.953642 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg"] Jun 02 20:16:28.955793 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:16:28.955768 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod865a7984_70fd_4b47_9318_97b13fe46c62.slice/crio-5a98304b1ba0b07522fcda45b2b9e61710483f47e42e7e2585b02aa68f9ec547 WatchSource:0}: Error finding container 5a98304b1ba0b07522fcda45b2b9e61710483f47e42e7e2585b02aa68f9ec547: Status 404 returned error can't find the container with id 5a98304b1ba0b07522fcda45b2b9e61710483f47e42e7e2585b02aa68f9ec547 Jun 02 20:16:29.238928 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:29.238892 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" event={"ID":"865a7984-70fd-4b47-9318-97b13fe46c62","Type":"ContainerStarted","Data":"5a98304b1ba0b07522fcda45b2b9e61710483f47e42e7e2585b02aa68f9ec547"} Jun 02 20:16:33.257316 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:33.257274 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" event={"ID":"865a7984-70fd-4b47-9318-97b13fe46c62","Type":"ContainerStarted","Data":"3729d87e3f3c58641230eea440f8a8168c8d40cb15b108d5726c504af0fd159e"} Jun 02 20:16:34.263625 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:34.263591 2577 generic.go:358] "Generic (PLEG): container finished" podID="865a7984-70fd-4b47-9318-97b13fe46c62" containerID="3729d87e3f3c58641230eea440f8a8168c8d40cb15b108d5726c504af0fd159e" exitCode=0 Jun 02 20:16:34.264005 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:34.263682 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" event={"ID":"865a7984-70fd-4b47-9318-97b13fe46c62","Type":"ContainerDied","Data":"3729d87e3f3c58641230eea440f8a8168c8d40cb15b108d5726c504af0fd159e"} Jun 02 20:16:36.272770 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:36.272735 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" event={"ID":"865a7984-70fd-4b47-9318-97b13fe46c62","Type":"ContainerStarted","Data":"0fe14d3017c8450764df2b80adb923c228de707ec822777cc2b4ec7f370e93ad"} Jun 02 20:16:36.301717 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:36.301655 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" podStartSLOduration=1.622528322 podStartE2EDuration="8.301640321s" podCreationTimestamp="2026-06-02 20:16:28 +0000 UTC" firstStartedPulling="2026-06-02 20:16:28.957562045 +0000 UTC m=+865.155124718" lastFinishedPulling="2026-06-02 20:16:35.636674027 +0000 UTC m=+871.834236717" observedRunningTime="2026-06-02 20:16:36.299977277 +0000 UTC m=+872.497539977" watchObservedRunningTime="2026-06-02 20:16:36.301640321 +0000 UTC m=+872.499203015" Jun 02 20:16:38.823446 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:38.823413 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:38.823841 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:38.823458 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:38.835800 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:38.835776 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:39.292356 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:39.292324 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg" Jun 02 20:16:42.582563 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.582532 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh"] Jun 02 20:16:42.591444 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.591419 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.594704 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.594525 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 02 20:16:42.595958 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.595936 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh"] Jun 02 20:16:42.676588 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.676561 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2m2h\" (UniqueName: \"kubernetes.io/projected/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-kube-api-access-c2m2h\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.676724 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.676593 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.676724 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.676659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.676724 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.676701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.676839 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.676730 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.676839 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.676747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.778186 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.778159 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.778336 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.778200 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.778336 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.778220 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.778472 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.778457 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c2m2h\" (UniqueName: \"kubernetes.io/projected/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-kube-api-access-c2m2h\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.778511 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.778479 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.778561 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.778512 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.778632 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.778608 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.778684 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.778633 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.778831 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.778798 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.780589 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.780571 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.780644 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.780580 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.787674 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.787651 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2m2h\" (UniqueName: \"kubernetes.io/projected/efabbcd0-b180-4d2d-b78f-7cc8d24cb07e-kube-api-access-c2m2h\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh\" (UID: \"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:42.903453 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:42.903359 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:16:43.037723 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:43.037699 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh"] Jun 02 20:16:43.040096 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:16:43.040067 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefabbcd0_b180_4d2d_b78f_7cc8d24cb07e.slice/crio-ebfd2b0ec5c47b4373bee5981461a66a42a43ea7aac2719fbfde0a266578d942 WatchSource:0}: Error finding container ebfd2b0ec5c47b4373bee5981461a66a42a43ea7aac2719fbfde0a266578d942: Status 404 returned error can't find the container with id ebfd2b0ec5c47b4373bee5981461a66a42a43ea7aac2719fbfde0a266578d942 Jun 02 20:16:43.296329 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:43.296296 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" event={"ID":"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e","Type":"ContainerStarted","Data":"88fee43f9e11eb34e230708ec7afd8327525e757e7dcf00593645fb5acdd5e29"} Jun 02 20:16:43.296329 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:43.296336 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" event={"ID":"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e","Type":"ContainerStarted","Data":"ebfd2b0ec5c47b4373bee5981461a66a42a43ea7aac2719fbfde0a266578d942"} Jun 02 20:16:57.682660 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.682574 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg"] Jun 02 20:16:57.686004 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.685969 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.688341 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.688317 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 02 20:16:57.695022 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.694979 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg"] Jun 02 20:16:57.810697 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.810665 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.810852 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.810715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.810852 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.810810 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.810852 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.810843 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnttj\" (UniqueName: \"kubernetes.io/projected/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-kube-api-access-mnttj\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.810969 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.810940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.811037 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.810966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.911631 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.911595 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.911631 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.911634 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.911845 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.911666 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.911845 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.911694 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.911845 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.911721 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.911845 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.911739 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mnttj\" (UniqueName: \"kubernetes.io/projected/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-kube-api-access-mnttj\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.912122 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.912089 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.912174 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.912094 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.912209 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.912187 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.914049 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.914028 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.914156 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.914139 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.919814 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.919795 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnttj\" (UniqueName: \"kubernetes.io/projected/9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1-kube-api-access-mnttj\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg\" (UID: \"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:57.998203 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:57.998140 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:16:58.127292 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:58.127259 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg"] Jun 02 20:16:58.127675 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:16:58.127641 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9336e0c8_da2c_48f9_9f3c_c1b4f1a3a4f1.slice/crio-b9bddbfe2288086d7c168d1d2993c6855063c7d4ea39110d1f62b966574b9d5a WatchSource:0}: Error finding container b9bddbfe2288086d7c168d1d2993c6855063c7d4ea39110d1f62b966574b9d5a: Status 404 returned error can't find the container with id b9bddbfe2288086d7c168d1d2993c6855063c7d4ea39110d1f62b966574b9d5a Jun 02 20:16:58.350695 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:58.350595 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" event={"ID":"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1","Type":"ContainerStarted","Data":"b642d28b25eaa02619c498384a5ff7a16416adbfc44bb11f768e57d681c1d325"} Jun 02 20:16:58.350695 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:16:58.350637 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" event={"ID":"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1","Type":"ContainerStarted","Data":"b9bddbfe2288086d7c168d1d2993c6855063c7d4ea39110d1f62b966574b9d5a"} Jun 02 20:17:03.370008 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:03.369961 2577 generic.go:358] "Generic (PLEG): container finished" podID="9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1" containerID="b642d28b25eaa02619c498384a5ff7a16416adbfc44bb11f768e57d681c1d325" exitCode=0 Jun 02 20:17:03.370365 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:03.370043 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" event={"ID":"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1","Type":"ContainerDied","Data":"b642d28b25eaa02619c498384a5ff7a16416adbfc44bb11f768e57d681c1d325"} Jun 02 20:17:04.358097 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:04.358068 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:17:04.361511 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:04.361486 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:17:04.374256 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:04.374231 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" event={"ID":"9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1","Type":"ContainerStarted","Data":"d21b0a6c40431463c4e603d3fde286300ad92dd8bbb5b4887bf7eb1af2a03a2c"} Jun 02 20:17:04.396772 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:04.396730 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" podStartSLOduration=7.086763218 podStartE2EDuration="7.396716889s" podCreationTimestamp="2026-06-02 20:16:57 +0000 UTC" firstStartedPulling="2026-06-02 20:17:03.37064194 +0000 UTC m=+899.568204614" lastFinishedPulling="2026-06-02 20:17:03.680595612 +0000 UTC m=+899.878158285" observedRunningTime="2026-06-02 20:17:04.395022576 +0000 UTC m=+900.592585265" watchObservedRunningTime="2026-06-02 20:17:04.396716889 +0000 UTC m=+900.594279633" Jun 02 20:17:07.998470 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:07.998437 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:17:07.998470 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:07.998478 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:17:08.010844 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:08.010816 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:17:08.398213 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:08.398142 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg" Jun 02 20:17:35.489661 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:35.489631 2577 generic.go:358] "Generic (PLEG): container finished" podID="efabbcd0-b180-4d2d-b78f-7cc8d24cb07e" containerID="88fee43f9e11eb34e230708ec7afd8327525e757e7dcf00593645fb5acdd5e29" exitCode=0 Jun 02 20:17:35.490018 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:35.489712 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" event={"ID":"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e","Type":"ContainerDied","Data":"88fee43f9e11eb34e230708ec7afd8327525e757e7dcf00593645fb5acdd5e29"} Jun 02 20:17:36.495473 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:36.495438 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" event={"ID":"efabbcd0-b180-4d2d-b78f-7cc8d24cb07e","Type":"ContainerStarted","Data":"b0e54e50911508c202f9cd12d62746ee81671eae32b6df1fa7b68e3905620291"} Jun 02 20:17:36.514301 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:36.514247 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" podStartSLOduration=54.212285114 podStartE2EDuration="54.514233371s" podCreationTimestamp="2026-06-02 20:16:42 +0000 UTC" firstStartedPulling="2026-06-02 20:17:35.490393005 +0000 UTC m=+931.687955679" lastFinishedPulling="2026-06-02 20:17:35.792341255 +0000 UTC m=+931.989903936" observedRunningTime="2026-06-02 20:17:36.511275185 +0000 UTC m=+932.708837879" watchObservedRunningTime="2026-06-02 20:17:36.514233371 +0000 UTC m=+932.711796068" Jun 02 20:17:42.904067 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:42.904031 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:17:42.904509 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:42.904182 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:17:42.916616 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:42.916592 2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:17:43.530556 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:17:43.530528 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh" Jun 02 20:19:08.307731 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.307701 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bb676c484-h5zrv"] Jun 02 20:19:08.308193 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.307962 2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7bb676c484-h5zrv" podUID="6617307f-6a73-4eab-9d1a-68a1caca4c38" containerName="manager" containerID="cri-o://9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6" gracePeriod=30 Jun 02 20:19:08.555282 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.555258 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb676c484-h5zrv" Jun 02 20:19:08.719695 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.719663 2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8z2h\" (UniqueName: \"kubernetes.io/projected/6617307f-6a73-4eab-9d1a-68a1caca4c38-kube-api-access-t8z2h\") pod \"6617307f-6a73-4eab-9d1a-68a1caca4c38\" (UID: \"6617307f-6a73-4eab-9d1a-68a1caca4c38\") " Jun 02 20:19:08.721672 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.721642 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6617307f-6a73-4eab-9d1a-68a1caca4c38-kube-api-access-t8z2h" (OuterVolumeSpecName: "kube-api-access-t8z2h") pod "6617307f-6a73-4eab-9d1a-68a1caca4c38" (UID: "6617307f-6a73-4eab-9d1a-68a1caca4c38"). InnerVolumeSpecName "kube-api-access-t8z2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 20:19:08.817387 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.814535 2577 generic.go:358] "Generic (PLEG): container finished" podID="6617307f-6a73-4eab-9d1a-68a1caca4c38" containerID="9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6" exitCode=0 Jun 02 20:19:08.817387 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.814630 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb676c484-h5zrv" event={"ID":"6617307f-6a73-4eab-9d1a-68a1caca4c38","Type":"ContainerDied","Data":"9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6"} Jun 02 20:19:08.817387 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.814660 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb676c484-h5zrv" event={"ID":"6617307f-6a73-4eab-9d1a-68a1caca4c38","Type":"ContainerDied","Data":"951f61cb7f4ba816f554d275e7859a8d8d9b4a625955660b830b8057f82ca6e3"} Jun 02 20:19:08.817387 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.814680 2577 scope.go:117] "RemoveContainer" containerID="9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6" Jun 02 20:19:08.817387 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.815064 2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb676c484-h5zrv" Jun 02 20:19:08.820383 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.820339 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8z2h\" (UniqueName: \"kubernetes.io/projected/6617307f-6a73-4eab-9d1a-68a1caca4c38-kube-api-access-t8z2h\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jun 02 20:19:08.825067 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.825048 2577 scope.go:117] "RemoveContainer" containerID="9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6" Jun 02 20:19:08.825324 ip-10-0-134-118 kubenswrapper[2577]: E0602 20:19:08.825305 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6\": container with ID starting with 9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6 not found: ID does not exist" containerID="9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6" Jun 02 20:19:08.825386 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.825331 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6"} err="failed to get container status \"9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6\": rpc error: code = NotFound desc = could not find container \"9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6\": container with ID starting with 9bdf770b77992ab742590a9e7e230f11446364d0f14d0f5c451cc7340cfe6aa6 not found: ID does not exist" Jun 02 20:19:08.841690 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.841668 2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bb676c484-h5zrv"] Jun 02 20:19:08.847760 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:08.847741 2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bb676c484-h5zrv"] Jun 02 20:19:10.440941 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:19:10.440905 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6617307f-6a73-4eab-9d1a-68a1caca4c38" path="/var/lib/kubelet/pods/6617307f-6a73-4eab-9d1a-68a1caca4c38/volumes" Jun 02 20:22:04.382320 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:22:04.382291 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:22:04.386536 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:22:04.386514 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:27:04.407926 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:27:04.407897 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:27:04.412621 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:27:04.412598 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:32:04.435949 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:32:04.435918 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:32:04.441610 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:32:04.441585 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:37:04.467911 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:37:04.467809 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:37:04.471976 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:37:04.470449 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:40:47.354478 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:47.354447 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-df756cc49-qjn2v_5f3b48fa-0b04-4322-9b42-84c1159cce02/maas-api/0.log" Jun 02 20:40:48.060545 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:48.060517 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b87b6c86-hjbdn_853d9549-b2d8-4c4b-98b4-750872603e09/copy-manifests/0.log" Jun 02 20:40:48.067683 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:48.067663 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b87b6c86-hjbdn_853d9549-b2d8-4c4b-98b4-750872603e09/manager/0.log" Jun 02 20:40:49.976921 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:49.976894 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-6mpz9_1c59d850-71c4-4b97-aa5f-6cec92bb74c1/kuadrant-console-plugin/0.log" Jun 02 20:40:50.478725 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:50.478696 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-wsml8_117331b3-1a4d-4d03-adab-dd178113c2cd/manager/0.log" Jun 02 20:40:51.091958 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:51.091915 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-78d7545c57-vdmfk_bd35c25e-7aea-4d00-a38b-8e162b884828/kube-auth-proxy/0.log" Jun 02 20:40:51.463682 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:51.463656 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-trk6s_3f320db7-015f-4b20-990a-5918ba804edb/payload-processing/0.log" Jun 02 20:40:51.939307 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:51.939276 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg_865a7984-70fd-4b47-9318-97b13fe46c62/storage-initializer/0.log" Jun 02 20:40:51.947785 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:51.947759 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-wd2wg_865a7984-70fd-4b47-9318-97b13fe46c62/main/0.log" Jun 02 20:40:52.201574 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:52.201499 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh_efabbcd0-b180-4d2d-b78f-7cc8d24cb07e/storage-initializer/0.log" Jun 02 20:40:52.209450 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:52.209426 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-xf4zh_efabbcd0-b180-4d2d-b78f-7cc8d24cb07e/main/0.log" Jun 02 20:40:52.567804 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:52.567732 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg_9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1/storage-initializer/0.log" Jun 02 20:40:52.576612 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:52.576589 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-shnfg_9336e0c8-da2c-48f9-9f3c-c1b4f1a3a4f1/main/0.log" Jun 02 20:40:59.857286 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:40:59.857253 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gcxhv_248f5d15-1445-407d-9962-2d88539c781e/global-pull-secret-syncer/0.log" Jun 02 20:41:00.008101 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:00.008069 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vxdnd_12056731-756a-476b-ae32-1bd1e4c3420b/konnectivity-agent/0.log" Jun 02 20:41:00.060552 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:00.060510 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-118.ec2.internal_029bda4e5c20ee6cc79d398e78cb1fc5/haproxy/0.log" Jun 02 20:41:05.124437 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:05.124365 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-6mpz9_1c59d850-71c4-4b97-aa5f-6cec92bb74c1/kuadrant-console-plugin/0.log" Jun 02 20:41:05.326014 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:05.325968 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-wsml8_117331b3-1a4d-4d03-adab-dd178113c2cd/manager/0.log" Jun 02 20:41:07.284382 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.284352 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6ff98f5777-89rld_c987865a-3e14-4421-914f-9bbefd1b113c/metrics-server/0.log" Jun 02 20:41:07.441871 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.441838 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nctx4_deaba9a1-8f8c-4625-a280-2f3fc55ae934/node-exporter/0.log" Jun 02 20:41:07.465958 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.465935 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nctx4_deaba9a1-8f8c-4625-a280-2f3fc55ae934/kube-rbac-proxy/0.log" Jun 02 20:41:07.489293 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.489272 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nctx4_deaba9a1-8f8c-4625-a280-2f3fc55ae934/init-textfile/0.log" Jun 02 20:41:07.614191 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.614111 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-kprxq_7e4ab581-ed5e-4429-a136-5c01fe9e4d26/kube-rbac-proxy-main/0.log" Jun 02 20:41:07.640006 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.639978 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-kprxq_7e4ab581-ed5e-4429-a136-5c01fe9e4d26/kube-rbac-proxy-self/0.log" Jun 02 20:41:07.662670 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.662651 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-kprxq_7e4ab581-ed5e-4429-a136-5c01fe9e4d26/openshift-state-metrics/0.log" Jun 02 20:41:07.696226 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.696194 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4df56e78-cbea-4eb8-b0c5-472f8626cfd7/prometheus/0.log" Jun 02 20:41:07.725540 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.725513 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4df56e78-cbea-4eb8-b0c5-472f8626cfd7/config-reloader/0.log" Jun 02 20:41:07.754911 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.754889 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4df56e78-cbea-4eb8-b0c5-472f8626cfd7/thanos-sidecar/0.log" Jun 02 20:41:07.785940 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.785912 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4df56e78-cbea-4eb8-b0c5-472f8626cfd7/kube-rbac-proxy-web/0.log" Jun 02 20:41:07.811271 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.811250 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4df56e78-cbea-4eb8-b0c5-472f8626cfd7/kube-rbac-proxy/0.log" Jun 02 20:41:07.833866 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.833847 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4df56e78-cbea-4eb8-b0c5-472f8626cfd7/kube-rbac-proxy-thanos/0.log" Jun 02 20:41:07.862120 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.862099 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4df56e78-cbea-4eb8-b0c5-472f8626cfd7/init-config-reloader/0.log" Jun 02 20:41:07.890669 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.890609 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-55v4s_8fc61135-d7b1-4fd1-9559-dbf7ec574e7c/prometheus-operator/0.log" Jun 02 20:41:07.911789 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.911771 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-55v4s_8fc61135-d7b1-4fd1-9559-dbf7ec574e7c/kube-rbac-proxy/0.log" Jun 02 20:41:07.945331 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:07.945315 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-xwjgx_39ee9e2a-0684-4e09-86bf-2cdb77db149c/prometheus-operator-admission-webhook/0.log" Jun 02 20:41:09.895738 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:09.895707 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/1.log" Jun 02 20:41:09.900570 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:09.900548 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-f2g7w_ecf92cc3-a384-405d-b806-b183462bb67a/console-operator/2.log" Jun 02 20:41:10.648920 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.648888 2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz"] Jun 02 20:41:10.649276 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.649261 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6617307f-6a73-4eab-9d1a-68a1caca4c38" containerName="manager" Jun 02 20:41:10.649321 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.649278 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6617307f-6a73-4eab-9d1a-68a1caca4c38" containerName="manager" Jun 02 20:41:10.649358 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.649342 2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="6617307f-6a73-4eab-9d1a-68a1caca4c38" containerName="manager" Jun 02 20:41:10.652594 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.652574 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.654857 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.654837 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fl6xc\"/\"kube-root-ca.crt\"" Jun 02 20:41:10.655506 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.655486 2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fl6xc\"/\"default-dockercfg-4c8mf\"" Jun 02 20:41:10.655625 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.655488 2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fl6xc\"/\"openshift-service-ca.crt\"" Jun 02 20:41:10.664185 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.664162 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz"] Jun 02 20:41:10.842922 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.842888 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-sys\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.843125 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.842940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-proc\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.843125 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.843023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn2hh\" (UniqueName: \"kubernetes.io/projected/df36646d-4171-4a0e-a881-515996535c22-kube-api-access-dn2hh\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.843125 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.843078 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-lib-modules\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.843125 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.843101 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-podres\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.944097 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.944071 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-proc\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.944510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.944118 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dn2hh\" (UniqueName: \"kubernetes.io/projected/df36646d-4171-4a0e-a881-515996535c22-kube-api-access-dn2hh\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.944510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.944157 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-lib-modules\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.944510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.944174 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-podres\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.944510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.944182 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-proc\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.944510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.944236 2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-sys\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.944510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.944298 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-sys\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.944510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.944308 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-lib-modules\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.944510 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.944350 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/df36646d-4171-4a0e-a881-515996535c22-podres\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.955587 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.955563 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn2hh\" (UniqueName: \"kubernetes.io/projected/df36646d-4171-4a0e-a881-515996535c22-kube-api-access-dn2hh\") pod \"perf-node-gather-daemonset-q6sbz\" (UID: \"df36646d-4171-4a0e-a881-515996535c22\") " pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:10.964096 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:10.964077 2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:11.105529 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:11.105494 2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz"] Jun 02 20:41:11.108571 ip-10-0-134-118 kubenswrapper[2577]: W0602 20:41:11.108537 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddf36646d_4171_4a0e_a881_515996535c22.slice/crio-e844849c757c3a04243ed9c2463f92e96f73f4cf0d395e24871ab477aaeb4b96 WatchSource:0}: Error finding container e844849c757c3a04243ed9c2463f92e96f73f4cf0d395e24871ab477aaeb4b96: Status 404 returned error can't find the container with id e844849c757c3a04243ed9c2463f92e96f73f4cf0d395e24871ab477aaeb4b96 Jun 02 20:41:11.110035 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:11.110018 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 20:41:11.302486 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:11.302458 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" event={"ID":"df36646d-4171-4a0e-a881-515996535c22","Type":"ContainerStarted","Data":"55e18f983934bb12bfa81a683dba2d4c36fc39837f0fc7eeac23949582e668f0"} Jun 02 20:41:11.302486 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:11.302492 2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" event={"ID":"df36646d-4171-4a0e-a881-515996535c22","Type":"ContainerStarted","Data":"e844849c757c3a04243ed9c2463f92e96f73f4cf0d395e24871ab477aaeb4b96"} Jun 02 20:41:11.302700 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:11.302553 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:11.321962 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:11.321903 2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" podStartSLOduration=1.321887427 podStartE2EDuration="1.321887427s" podCreationTimestamp="2026-06-02 20:41:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 20:41:11.321157349 +0000 UTC m=+2347.518720044" watchObservedRunningTime="2026-06-02 20:41:11.321887427 +0000 UTC m=+2347.519450124" Jun 02 20:41:11.920938 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:11.920910 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zkhfn_e95b758c-5d94-4f5b-83cd-cbb132ef586e/dns/0.log" Jun 02 20:41:11.942380 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:11.942348 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zkhfn_e95b758c-5d94-4f5b-83cd-cbb132ef586e/kube-rbac-proxy/0.log" Jun 02 20:41:11.972330 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:11.972291 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6scmv_221c0697-6afa-4a37-902f-3048a712c075/dns-node-resolver/0.log" Jun 02 20:41:12.534297 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:12.534269 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sdkqb_5089440b-e3f8-4581-8d0d-03c318033288/node-ca/0.log" Jun 02 20:41:13.541567 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:13.541540 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-78d7545c57-vdmfk_bd35c25e-7aea-4d00-a38b-8e162b884828/kube-auth-proxy/0.log" Jun 02 20:41:13.691248 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:13.691216 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-trk6s_3f320db7-015f-4b20-990a-5918ba804edb/payload-processing/0.log" Jun 02 20:41:14.347034 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:14.346986 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cdhs6_2f0a22d1-2b80-4b90-ba52-b598b876a693/serve-healthcheck-canary/0.log" Jun 02 20:41:14.940414 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:14.940386 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nxgbz_f95f22ec-1657-4507-af20-664bf4e3f7d3/kube-rbac-proxy/0.log" Jun 02 20:41:14.969352 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:14.969327 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nxgbz_f95f22ec-1657-4507-af20-664bf4e3f7d3/exporter/0.log" Jun 02 20:41:14.993454 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:14.993421 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nxgbz_f95f22ec-1657-4507-af20-664bf4e3f7d3/extractor/0.log" Jun 02 20:41:17.315796 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:17.315771 2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fl6xc/perf-node-gather-daemonset-q6sbz" Jun 02 20:41:17.380106 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:17.380077 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-df756cc49-qjn2v_5f3b48fa-0b04-4322-9b42-84c1159cce02/maas-api/0.log" Jun 02 20:41:17.686640 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:17.686598 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b87b6c86-hjbdn_853d9549-b2d8-4c4b-98b4-750872603e09/manager/0.log" Jun 02 20:41:17.715626 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:17.715600 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b87b6c86-hjbdn_853d9549-b2d8-4c4b-98b4-750872603e09/copy-manifests/0.log" Jun 02 20:41:19.243275 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:19.243248 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f89f75957-rndbj_bb1d5e4a-9184-4235-ad94-d4fee8ba93cf/manager/0.log" Jun 02 20:41:24.357127 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:24.357096 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-25l5c_b1ab5cf1-3968-421f-b574-74aef77fd2b5/migrator/0.log" Jun 02 20:41:24.386378 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:24.386356 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-25l5c_b1ab5cf1-3968-421f-b574-74aef77fd2b5/graceful-termination/0.log" Jun 02 20:41:24.786362 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:24.786313 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-twl68_0b220d09-f221-4b6f-935e-58a4d367abd3/kube-storage-version-migrator-operator/1.log" Jun 02 20:41:24.787169 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:24.787128 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-twl68_0b220d09-f221-4b6f-935e-58a4d367abd3/kube-storage-version-migrator-operator/0.log" Jun 02 20:41:25.927307 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:25.927278 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5gfl5_79ca1d6a-f410-48c0-8217-adef4c71ea5d/kube-multus/0.log" Jun 02 20:41:26.467054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:26.467027 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vthxs_4bd9970a-116d-4749-8219-ed8d4f5be7f2/kube-multus-additional-cni-plugins/0.log" Jun 02 20:41:26.488270 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:26.488243 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vthxs_4bd9970a-116d-4749-8219-ed8d4f5be7f2/egress-router-binary-copy/0.log" Jun 02 20:41:26.509663 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:26.509640 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vthxs_4bd9970a-116d-4749-8219-ed8d4f5be7f2/cni-plugins/0.log" Jun 02 20:41:26.533810 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:26.533790 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vthxs_4bd9970a-116d-4749-8219-ed8d4f5be7f2/bond-cni-plugin/0.log" Jun 02 20:41:26.556935 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:26.556919 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vthxs_4bd9970a-116d-4749-8219-ed8d4f5be7f2/routeoverride-cni/0.log" Jun 02 20:41:26.579266 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:26.579246 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vthxs_4bd9970a-116d-4749-8219-ed8d4f5be7f2/whereabouts-cni-bincopy/0.log" Jun 02 20:41:26.600244 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:26.600224 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vthxs_4bd9970a-116d-4749-8219-ed8d4f5be7f2/whereabouts-cni/0.log" Jun 02 20:41:26.784802 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:26.784734 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q8xf6_d34cf515-d3ea-4298-97e9-3b98e1963700/network-metrics-daemon/0.log" Jun 02 20:41:26.808705 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:26.808681 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q8xf6_d34cf515-d3ea-4298-97e9-3b98e1963700/kube-rbac-proxy/0.log" Jun 02 20:41:27.684920 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:27.684887 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmkdm_5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd/ovn-controller/0.log" Jun 02 20:41:27.718720 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:27.718695 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmkdm_5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd/ovn-acl-logging/0.log" Jun 02 20:41:27.743335 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:27.743312 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmkdm_5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd/kube-rbac-proxy-node/0.log" Jun 02 20:41:27.786886 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:27.786863 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmkdm_5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 20:41:27.824606 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:27.824585 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmkdm_5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd/northd/0.log" Jun 02 20:41:27.857054 ip-10-0-134-118 kubenswrapper[2577]: I0602 20:41:27.857032 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jmkdm_5d27d5f6-05ea-4b7f-86de-3c39cdf42bcd/nbdb/0.log"