Jun 01 20:33:59.186721 ip-10-0-141-183 systemd[1]: Starting Kubernetes Kubelet... Jun 01 20:33:59.574647 ip-10-0-141-183 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 01 20:33:59.574647 ip-10-0-141-183 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 01 20:33:59.574647 ip-10-0-141-183 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 01 20:33:59.574647 ip-10-0-141-183 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 01 20:33:59.574647 ip-10-0-141-183 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 01 20:33:59.575955 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.575868 2559 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 01 20:33:59.579300 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579283 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 20:33:59.579300 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579300 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579304 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579308 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579311 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579314 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579317 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579320 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579323 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579326 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579329 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579334 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579338 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579341 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579344 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579347 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579350 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579353 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579355 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579358 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579361 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 20:33:59.579362 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579364 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579367 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579369 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579372 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579375 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579378 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579381 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579384 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579386 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579389 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579391 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579394 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579396 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579399 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579401 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579404 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579406 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579409 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579411 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579414 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 20:33:59.579867 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579417 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579431 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579434 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579436 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579438 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579441 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579444 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579446 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579449 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579451 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579454 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579457 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579462 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579464 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579468 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579471 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579474 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579476 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579479 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 20:33:59.580353 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579482 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579485 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579487 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579490 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579494 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579497 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579499 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579502 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579504 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579507 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579509 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579512 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579514 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579517 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579519 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579524 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579526 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579529 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579531 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 20:33:59.580836 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579534 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579537 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579539 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579542 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579544 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579547 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579549 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579968 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579975 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579978 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579981 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579983 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579986 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579989 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579992 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579996 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.579999 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580002 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580004 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 20:33:59.581303 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580007 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580010 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580013 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580017 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580020 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580023 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580026 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580029 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580032 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580036 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580039 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580042 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580045 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580048 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580051 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580054 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580058 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580061 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580064 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 20:33:59.581780 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580066 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580069 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580072 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580074 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580077 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580080 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580082 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580085 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580088 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580090 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580093 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580095 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580098 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580100 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580103 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580105 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580108 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580110 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580112 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580115 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 20:33:59.582277 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580117 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580120 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580122 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580125 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580128 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580130 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580133 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580135 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580138 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580141 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580144 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580147 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580150 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580153 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580155 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580158 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580160 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580163 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580165 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580168 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 20:33:59.582782 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580170 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580173 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580175 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580178 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580180 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580183 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580186 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580189 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580191 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580194 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580196 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580198 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580201 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580203 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.580206 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581263 2559 flags.go:64] FLAG: --address="0.0.0.0" Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581273 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581279 2559 flags.go:64] FLAG: --anonymous-auth="true" Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581284 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581293 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581297 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 01 20:33:59.583267 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581301 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581306 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581310 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581313 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581317 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581320 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581323 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581327 2559 flags.go:64] FLAG: --cgroup-root="" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581329 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581333 2559 flags.go:64] FLAG: --client-ca-file="" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581336 2559 flags.go:64] FLAG: --cloud-config="" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581339 2559 flags.go:64] FLAG: --cloud-provider="external" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581342 2559 flags.go:64] FLAG: --cluster-dns="[]" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581347 2559 flags.go:64] FLAG: --cluster-domain="" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581350 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581353 2559 flags.go:64] FLAG: --config-dir="" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581356 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581359 2559 flags.go:64] FLAG: --container-log-max-files="5" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581363 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581367 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581370 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581373 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581376 2559 flags.go:64] FLAG: --contention-profiling="false" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581379 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 01 20:33:59.583805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581382 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581385 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581389 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581393 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581397 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581400 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581403 2559 flags.go:64] FLAG: --enable-load-reader="false" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581406 2559 flags.go:64] FLAG: --enable-server="true" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581409 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581413 2559 flags.go:64] FLAG: --event-burst="100" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581416 2559 flags.go:64] FLAG: --event-qps="50" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581432 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581436 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581439 2559 flags.go:64] FLAG: --eviction-hard="" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581443 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581446 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581452 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581455 2559 flags.go:64] FLAG: --eviction-soft="" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581458 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581461 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581464 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581466 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581469 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581472 2559 flags.go:64] FLAG: --fail-swap-on="true" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581475 2559 flags.go:64] FLAG: --feature-gates="" Jun 01 20:33:59.584502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581479 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581482 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581486 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581489 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581492 2559 flags.go:64] FLAG: --healthz-port="10248" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581494 2559 flags.go:64] FLAG: --help="false" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581497 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581500 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581504 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581507 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581510 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581514 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581518 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581521 2559 flags.go:64] FLAG: --image-service-endpoint="" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581524 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581527 2559 flags.go:64] FLAG: --kube-api-burst="100" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581530 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581533 2559 flags.go:64] FLAG: --kube-api-qps="50" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581536 2559 flags.go:64] FLAG: --kube-reserved="" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581539 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581542 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581544 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581547 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581551 2559 flags.go:64] FLAG: --lock-file="" Jun 01 20:33:59.585101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581554 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581557 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581560 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581565 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581568 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581571 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581574 2559 flags.go:64] FLAG: --logging-format="text" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581577 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581580 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581583 2559 flags.go:64] FLAG: --manifest-url="" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581586 2559 flags.go:64] FLAG: --manifest-url-header="" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581590 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581593 2559 flags.go:64] FLAG: --max-open-files="1000000" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581597 2559 flags.go:64] FLAG: --max-pods="110" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581600 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581603 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581605 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581608 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581611 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581614 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581617 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581626 2559 flags.go:64] FLAG: --node-status-max-images="50" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581629 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581632 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jun 01 20:33:59.585697 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581635 2559 flags.go:64] FLAG: --pod-cidr="" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581638 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581644 2559 flags.go:64] FLAG: --pod-manifest-path="" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581647 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581650 2559 flags.go:64] FLAG: --pods-per-core="0" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581653 2559 flags.go:64] FLAG: --port="10250" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581656 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581660 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0805a147115699f43" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581663 2559 flags.go:64] FLAG: --qos-reserved="" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581666 2559 flags.go:64] FLAG: --read-only-port="10255" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581669 2559 flags.go:64] FLAG: --register-node="true" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581672 2559 flags.go:64] FLAG: --register-schedulable="true" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581675 2559 flags.go:64] FLAG: --register-with-taints="" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581679 2559 flags.go:64] FLAG: --registry-burst="10" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581682 2559 flags.go:64] FLAG: --registry-qps="5" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581685 2559 flags.go:64] FLAG: --reserved-cpus="" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581688 2559 flags.go:64] FLAG: --reserved-memory="" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581692 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581695 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581698 2559 flags.go:64] FLAG: --rotate-certificates="false" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581701 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581704 2559 flags.go:64] FLAG: --runonce="false" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581707 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581710 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581713 2559 flags.go:64] FLAG: --seccomp-default="false" Jun 01 20:33:59.586276 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581716 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581719 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581722 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581725 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581728 2559 flags.go:64] FLAG: --storage-driver-password="root" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581732 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581735 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581738 2559 flags.go:64] FLAG: --storage-driver-user="root" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581741 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581744 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581747 2559 flags.go:64] FLAG: --system-cgroups="" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581750 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581756 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581758 2559 flags.go:64] FLAG: --tls-cert-file="" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581763 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581767 2559 flags.go:64] FLAG: --tls-min-version="" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581770 2559 flags.go:64] FLAG: --tls-private-key-file="" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581773 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581776 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581779 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581782 2559 flags.go:64] FLAG: --v="2" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581786 2559 flags.go:64] FLAG: --version="false" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581791 2559 flags.go:64] FLAG: --vmodule="" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581795 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.581799 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 01 20:33:59.586892 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581894 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581898 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581901 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581904 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581907 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581909 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581913 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581916 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581919 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581921 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581924 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581926 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581929 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581932 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581935 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581937 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581940 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581943 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581945 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581948 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 20:33:59.587587 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581950 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581954 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581956 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581959 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581961 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581964 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581966 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581968 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581971 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581973 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581976 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581979 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581981 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581985 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581989 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581992 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581995 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.581998 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582000 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 20:33:59.588097 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582006 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582009 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582011 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582014 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582017 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582019 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582022 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582025 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582028 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582030 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582033 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582036 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582039 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582041 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582044 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582047 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582049 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582053 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582057 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582059 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 20:33:59.588592 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582062 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582065 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582068 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582070 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582073 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582076 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582078 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582081 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582084 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582086 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582088 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582091 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582096 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582098 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582101 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582103 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582106 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582109 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582111 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582114 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 20:33:59.589096 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582117 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 20:33:59.589672 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582119 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 20:33:59.589672 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582122 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 20:33:59.589672 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582125 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 20:33:59.589672 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582127 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 20:33:59.589672 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582129 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 20:33:59.589672 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.582132 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 20:33:59.589672 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.582751 2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 01 20:33:59.592003 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.591980 2559 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 01 20:33:59.592003 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.592000 2559 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592052 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592057 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592061 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592064 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592067 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592070 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592073 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592075 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592078 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592081 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592084 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592086 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592089 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592092 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592095 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592097 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592100 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592103 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592105 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 20:33:59.592129 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592108 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592113 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592118 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592121 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592124 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592127 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592130 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592132 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592135 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592138 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592141 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592144 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592146 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592149 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592152 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592154 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592157 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592159 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592163 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 20:33:59.592646 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592167 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592169 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592172 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592175 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592177 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592180 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592183 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592186 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592188 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592191 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592194 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592196 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592198 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592201 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592203 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592207 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592209 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592212 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592215 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 20:33:59.593127 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592217 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592220 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592222 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592225 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592228 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592231 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592233 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592236 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592238 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592241 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592243 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592246 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592248 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592250 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592253 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592256 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592258 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592261 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592264 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592267 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 20:33:59.593725 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592270 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592272 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592275 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592277 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592280 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592282 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592285 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592288 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592291 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.592296 2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592389 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592395 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592398 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592400 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592403 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592406 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 20:33:59.594217 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592409 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592412 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592414 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592417 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592436 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592439 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592442 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592444 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592447 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592449 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592452 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592455 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592457 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592460 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592462 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592465 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592468 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592471 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592473 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592476 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 20:33:59.594632 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592478 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592481 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592484 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592487 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592489 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592492 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592495 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592498 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592501 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592503 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592506 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592508 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592511 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592514 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592516 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592519 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592521 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592524 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592526 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592529 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 20:33:59.595113 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592531 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592534 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592537 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592539 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592542 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592544 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592547 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592549 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592552 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592555 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592558 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592561 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592563 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592567 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592570 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592573 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592577 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592580 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592583 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 20:33:59.595626 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592586 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592589 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592592 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592595 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592597 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592600 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592603 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592605 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592608 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592610 2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592613 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592615 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592618 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592621 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592623 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592626 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592628 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592631 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592633 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 20:33:59.596084 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592636 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 20:33:59.596575 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:33:59.592638 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 20:33:59.596575 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.592644 2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 01 20:33:59.596575 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.593291 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jun 01 20:33:59.596575 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.596501 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 01 20:33:59.597287 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.597276 2559 server.go:1019] "Starting client certificate rotation" Jun 01 20:33:59.597389 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.597370 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 01 20:33:59.597436 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.597406 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 01 20:33:59.619012 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.618993 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 01 20:33:59.620929 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.620907 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 01 20:33:59.635023 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.635000 2559 log.go:25] "Validated CRI v1 runtime API" Jun 01 20:33:59.640702 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.640686 2559 log.go:25] "Validated CRI v1 image API" Jun 01 20:33:59.641893 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.641876 2559 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 01 20:33:59.644301 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.644278 2559 fs.go:135] Filesystem UUIDs: map[6273843b-8cf9-400a-8c20-564f5b6ed43c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e4599e78-8073-461c-8ba6-2cc30484ccd8:/dev/nvme0n1p4] Jun 01 20:33:59.644374 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.644298 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 01 20:33:59.649013 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.648993 2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 01 20:33:59.650621 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.650498 2559 manager.go:217] Machine: {Timestamp:2026-06-01 20:33:59.64993378 +0000 UTC m=+0.357306308 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100285 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24cb9641bc477e2d06fc7ba7089d73 SystemUUID:ec24cb96-41bc-477e-2d06-fc7ba7089d73 BootID:2c787488-918a-436e-9cc2-a849a5e6da10 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e5:02:0e:b2:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e5:02:0e:b2:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:e1:71:77:70:1c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 01 20:33:59.651915 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.651900 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 01 20:33:59.652040 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.652023 2559 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 01 20:33:59.654732 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.654705 2559 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 01 20:33:59.654909 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.654736 2559 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-183.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 01 20:33:59.654994 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.654924 2559 topology_manager.go:138] "Creating topology manager with none policy" Jun 01 20:33:59.654994 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.654936 2559 container_manager_linux.go:306] "Creating device plugin manager" Jun 01 20:33:59.654994 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.654946 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 01 20:33:59.655701 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.655689 2559 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 01 20:33:59.656674 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.656661 2559 state_mem.go:36] "Initialized new in-memory state store" Jun 01 20:33:59.656802 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.656792 2559 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 01 20:33:59.658823 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.658811 2559 kubelet.go:491] "Attempting to sync node with API server" Jun 01 20:33:59.658879 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.658829 2559 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 01 20:33:59.658879 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.658846 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 01 20:33:59.658879 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.658860 2559 kubelet.go:397] "Adding apiserver pod source" Jun 01 20:33:59.658879 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.658872 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 01 20:33:59.659976 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.659962 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 01 20:33:59.660055 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.659985 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 01 20:33:59.662584 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.662554 2559 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 01 20:33:59.664504 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.664491 2559 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 01 20:33:59.666592 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666568 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 01 20:33:59.666592 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666587 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 01 20:33:59.666592 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666594 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666601 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666612 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666617 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666623 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666629 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666638 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666644 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666653 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666658 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 01 20:33:59.666703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.666689 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4zk6k" Jun 01 20:33:59.667537 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.667528 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 01 20:33:59.667537 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.667538 2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 01 20:33:59.670797 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.670783 2559 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 01 20:33:59.670861 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.670820 2559 server.go:1295] "Started kubelet" Jun 01 20:33:59.670993 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.670941 2559 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 01 20:33:59.671050 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.671024 2559 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 01 20:33:59.671095 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.671000 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 01 20:33:59.671665 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.671648 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4zk6k" Jun 01 20:33:59.671709 ip-10-0-141-183 systemd[1]: Started Kubernetes Kubelet. Jun 01 20:33:59.672050 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.671948 2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-183.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 01 20:33:59.672134 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.672098 2559 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-183.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 01 20:33:59.672332 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.672169 2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 01 20:33:59.673998 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.673982 2559 server.go:317] "Adding debug handlers to kubelet server" Jun 01 20:33:59.674742 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.674723 2559 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 01 20:33:59.678001 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.677185 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-183.ec2.internal.18b50f4a57d9f147 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-183.ec2.internal,UID:ip-10-0-141-183.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-183.ec2.internal,},FirstTimestamp:2026-06-01 20:33:59.670796615 +0000 UTC m=+0.378169143,LastTimestamp:2026-06-01 20:33:59.670796615 +0000 UTC m=+0.378169143,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-183.ec2.internal,}" Jun 01 20:33:59.678373 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.678341 2559 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 01 20:33:59.679669 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.679654 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 01 20:33:59.679669 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.679667 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 01 20:33:59.679781 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.679701 2559 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 01 20:33:59.679781 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.679709 2559 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 01 20:33:59.679893 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.679793 2559 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 01 20:33:59.679893 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.679827 2559 reconstruct.go:97] "Volume reconstruction finished" Jun 01 20:33:59.679893 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.679832 2559 reconciler.go:26] "Reconciler: start to sync state" Jun 01 20:33:59.680325 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.680305 2559 factory.go:55] Registering systemd factory Jun 01 20:33:59.680325 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.680318 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:33:59.680494 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.680327 2559 factory.go:223] Registration of the systemd container factory successfully Jun 01 20:33:59.680543 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.680536 2559 factory.go:153] Registering CRI-O factory Jun 01 20:33:59.680585 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.680553 2559 factory.go:223] Registration of the crio container factory successfully Jun 01 20:33:59.680656 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.680627 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 01 20:33:59.680656 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.680646 2559 factory.go:103] Registering Raw factory Jun 01 20:33:59.680752 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.680665 2559 manager.go:1196] Started watching for new ooms in manager Jun 01 20:33:59.681115 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.681097 2559 manager.go:319] Starting recovery of all containers Jun 01 20:33:59.685117 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.684921 2559 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 20:33:59.686331 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.686303 2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-183.ec2.internal\" not found" node="ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.694823 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.694806 2559 manager.go:324] Recovery completed Jun 01 20:33:59.699105 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.699092 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 20:33:59.700949 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.700935 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientMemory" Jun 01 20:33:59.700997 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.700963 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 20:33:59.700997 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.700976 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientPID" Jun 01 20:33:59.701458 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.701444 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 01 20:33:59.701458 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.701457 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 01 20:33:59.701580 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.701481 2559 state_mem.go:36] "Initialized new in-memory state store" Jun 01 20:33:59.703992 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.703980 2559 policy_none.go:49] "None policy: Start" Jun 01 20:33:59.704052 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.703996 2559 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 01 20:33:59.704052 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.704011 2559 state_mem.go:35] "Initializing new in-memory state store" Jun 01 20:33:59.739229 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.739078 2559 manager.go:341] "Starting Device Plugin manager" Jun 01 20:33:59.739229 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.739128 2559 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 01 20:33:59.739229 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.739141 2559 server.go:85] "Starting device plugin registration server" Jun 01 20:33:59.739488 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.739473 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 01 20:33:59.739563 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.739496 2559 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 01 20:33:59.739622 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.739590 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 01 20:33:59.740086 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.739698 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 01 20:33:59.740086 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.739714 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 01 20:33:59.740203 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.740159 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 01 20:33:59.740203 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.740192 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:33:59.825066 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.824975 2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 01 20:33:59.826308 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.826281 2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 01 20:33:59.826444 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.826314 2559 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 01 20:33:59.826444 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.826340 2559 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 01 20:33:59.826444 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.826351 2559 kubelet.go:2451] "Starting kubelet main sync loop" Jun 01 20:33:59.826444 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.826395 2559 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 01 20:33:59.828601 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.828581 2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 20:33:59.840221 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.840204 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 20:33:59.841237 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.841219 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientMemory" Jun 01 20:33:59.841305 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.841252 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 20:33:59.841305 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.841267 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientPID" Jun 01 20:33:59.841305 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.841289 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.845948 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.845933 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.845998 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.845955 2559 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-183.ec2.internal\": node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:33:59.864805 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.864787 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:33:59.927344 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.927304 2559 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal"] Jun 01 20:33:59.927468 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.927412 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 20:33:59.930344 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.930328 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientMemory" Jun 01 20:33:59.930416 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.930357 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 20:33:59.930416 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.930372 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientPID" Jun 01 20:33:59.931609 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.931597 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 20:33:59.931770 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.931756 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.931807 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.931785 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 20:33:59.932635 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.932587 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientMemory" Jun 01 20:33:59.932635 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.932614 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 20:33:59.932635 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.932624 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientPID" Jun 01 20:33:59.932776 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.932588 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientMemory" Jun 01 20:33:59.932776 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.932683 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 20:33:59.932776 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.932695 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientPID" Jun 01 20:33:59.933765 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.933749 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.933852 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.933770 2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 20:33:59.934923 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.934906 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientMemory" Jun 01 20:33:59.935023 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.934936 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 20:33:59.935023 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.934946 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeHasSufficientPID" Jun 01 20:33:59.954109 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.954085 2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-183.ec2.internal\" not found" node="ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.958511 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.958496 2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-183.ec2.internal\" not found" node="ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.965522 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:33:59.965508 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:33:59.981904 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.981882 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab6fe5603972050dfc53ab2a7e03a2b3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal\" (UID: \"ab6fe5603972050dfc53ab2a7e03a2b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.981985 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.981909 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab6fe5603972050dfc53ab2a7e03a2b3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal\" (UID: \"ab6fe5603972050dfc53ab2a7e03a2b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" Jun 01 20:33:59.981985 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:33:59.981927 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b297c8edefffe884478fc112206a107d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-183.ec2.internal\" (UID: \"b297c8edefffe884478fc112206a107d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.066092 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:00.066069 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:34:00.082460 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.082394 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab6fe5603972050dfc53ab2a7e03a2b3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal\" (UID: \"ab6fe5603972050dfc53ab2a7e03a2b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.082460 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.082411 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab6fe5603972050dfc53ab2a7e03a2b3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal\" (UID: \"ab6fe5603972050dfc53ab2a7e03a2b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.082460 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.082449 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab6fe5603972050dfc53ab2a7e03a2b3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal\" (UID: \"ab6fe5603972050dfc53ab2a7e03a2b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.082601 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.082470 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b297c8edefffe884478fc112206a107d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-183.ec2.internal\" (UID: \"b297c8edefffe884478fc112206a107d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.082601 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.082492 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b297c8edefffe884478fc112206a107d-config\") pod \"kube-apiserver-proxy-ip-10-0-141-183.ec2.internal\" (UID: \"b297c8edefffe884478fc112206a107d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.082601 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.082522 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab6fe5603972050dfc53ab2a7e03a2b3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal\" (UID: \"ab6fe5603972050dfc53ab2a7e03a2b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.166858 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:00.166824 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:34:00.256396 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.256348 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.261999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.261979 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.267782 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:00.267766 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:34:00.368367 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:00.368280 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:34:00.468744 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:00.468724 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:34:00.569271 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:00.569237 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:34:00.597711 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.597680 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 01 20:34:00.598365 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.597842 2559 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 01 20:34:00.598365 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.597845 2559 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 01 20:34:00.670273 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:00.670242 2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-183.ec2.internal\" not found" Jun 01 20:34:00.676381 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.676349 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-31 20:28:59 +0000 UTC" deadline="2027-12-18 15:05:47.97237035 +0000 UTC" Jun 01 20:34:00.676381 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.676377 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13554h31m47.295996967s" Jun 01 20:34:00.679767 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.679745 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 01 20:34:00.690483 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.690463 2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 01 20:34:00.709995 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.709977 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-j957s" Jun 01 20:34:00.714827 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:00.714792 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6fe5603972050dfc53ab2a7e03a2b3.slice/crio-90526a22df5143591d759d4565c4b0861cc1baf5f44a8a953fc7531089b5cfa0 WatchSource:0}: Error finding container 90526a22df5143591d759d4565c4b0861cc1baf5f44a8a953fc7531089b5cfa0: Status 404 returned error can't find the container with id 90526a22df5143591d759d4565c4b0861cc1baf5f44a8a953fc7531089b5cfa0 Jun 01 20:34:00.715063 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:00.715042 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb297c8edefffe884478fc112206a107d.slice/crio-cf4e71f249f4db71a99a720fb6543f4216d0bfbb29403af3a46430d5d224369f WatchSource:0}: Error finding container cf4e71f249f4db71a99a720fb6543f4216d0bfbb29403af3a46430d5d224369f: Status 404 returned error can't find the container with id cf4e71f249f4db71a99a720fb6543f4216d0bfbb29403af3a46430d5d224369f Jun 01 20:34:00.716894 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.716876 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-j957s" Jun 01 20:34:00.718816 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.718804 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 01 20:34:00.740464 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.740416 2559 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 20:34:00.780444 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.780404 2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.787048 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.787028 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 01 20:34:00.788772 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.788759 2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" Jun 01 20:34:00.798011 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.797993 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 01 20:34:00.804386 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.804368 2559 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 20:34:00.829745 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.829701 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" event={"ID":"ab6fe5603972050dfc53ab2a7e03a2b3","Type":"ContainerStarted","Data":"90526a22df5143591d759d4565c4b0861cc1baf5f44a8a953fc7531089b5cfa0"} Jun 01 20:34:00.830688 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:00.830664 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal" event={"ID":"b297c8edefffe884478fc112206a107d","Type":"ContainerStarted","Data":"cf4e71f249f4db71a99a720fb6543f4216d0bfbb29403af3a46430d5d224369f"} Jun 01 20:34:01.660241 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.660195 2559 apiserver.go:52] "Watching apiserver" Jun 01 20:34:01.665189 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.665159 2559 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 01 20:34:01.667030 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.667003 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-z6xn6","openshift-network-diagnostics/network-check-target-p6w46","openshift-ovn-kubernetes/ovnkube-node-knvtx","kube-system/konnectivity-agent-9z8pf","kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rw9bx","openshift-multus/multus-q95l8","openshift-network-operator/iptables-alerter-bjf2s","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz","openshift-dns/node-resolver-xp92g","openshift-image-registry/node-ca-cx8lv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal","openshift-multus/multus-additional-cni-plugins-pd28x"] Jun 01 20:34:01.668435 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.668395 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.669524 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.669500 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:01.669628 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:01.669577 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:01.670528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.670283 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 01 20:34:01.670528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.670327 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 01 20:34:01.670528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.670349 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 01 20:34:01.670528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.670366 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5rbb5\"" Jun 01 20:34:01.670528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.670333 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 01 20:34:01.671945 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.671925 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.672040 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.671952 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:01.673117 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.673101 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.673626 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.673571 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 01 20:34:01.673752 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.673734 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 01 20:34:01.673864 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.673842 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 01 20:34:01.673929 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.673871 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 01 20:34:01.674516 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.674121 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 01 20:34:01.674516 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.674133 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bcpp2\"" Jun 01 20:34:01.674516 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.674128 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 01 20:34:01.674516 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.674232 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 01 20:34:01.674516 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.674285 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:01.674516 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.674316 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 01 20:34:01.674516 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:01.674379 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:01.674516 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.674399 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c6vrv\"" Jun 01 20:34:01.675805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.675380 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hhn55\"" Jun 01 20:34:01.675805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.675563 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 01 20:34:01.675805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.675798 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 01 20:34:01.678224 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.677952 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.679882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.679858 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.680253 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.680234 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-q8k58\"" Jun 01 20:34:01.680343 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.680325 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 01 20:34:01.680411 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.680396 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 01 20:34:01.680490 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.680453 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 01 20:34:01.681020 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.680971 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.681801 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.681586 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h97q2\"" Jun 01 20:34:01.681801 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.681680 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 01 20:34:01.681801 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.681728 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 01 20:34:01.681801 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.681745 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 01 20:34:01.682337 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.682310 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 01 20:34:01.682435 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.682341 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.682685 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.682594 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 01 20:34:01.682873 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.682853 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2v79b\"" Jun 01 20:34:01.683598 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.683572 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.684384 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.684363 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 01 20:34:01.684492 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.684468 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 01 20:34:01.684555 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.684489 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zvwhd\"" Jun 01 20:34:01.684555 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.684473 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 01 20:34:01.684937 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.684920 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 01 20:34:01.685094 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.685079 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g2hnl\"" Jun 01 20:34:01.685369 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.685339 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 01 20:34:01.690663 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690644 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-kubelet\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.690755 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690678 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-kubernetes\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.690755 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690709 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:01.690755 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690749 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86c772bf-658d-41c4-81e4-f4642d3e47d6-ovnkube-script-lib\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.690921 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690778 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-cni-bin\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.690921 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690819 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-var-lib-kubelet\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.690921 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690850 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92m8x\" (UniqueName: \"kubernetes.io/projected/d3172bfd-9aca-4acd-adf9-40803933a7eb-kube-api-access-92m8x\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.690921 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690885 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5979744-4da4-40e1-9001-477372c8d2c0-hosts-file\") pod \"node-resolver-xp92g\" (UID: \"d5979744-4da4-40e1-9001-477372c8d2c0\") " pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.690921 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690914 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-var-lib-openvswitch\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691147 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690937 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86c772bf-658d-41c4-81e4-f4642d3e47d6-ovnkube-config\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691147 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.690985 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-cni-binary-copy\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691147 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-systemd-units\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691147 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691070 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk87c\" (UniqueName: \"kubernetes.io/projected/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-kube-api-access-qk87c\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.691147 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691095 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-var-lib-kubelet\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691147 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691121 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.691147 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691144 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5979744-4da4-40e1-9001-477372c8d2c0-tmp-dir\") pod \"node-resolver-xp92g\" (UID: \"d5979744-4da4-40e1-9001-477372c8d2c0\") " pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691168 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691194 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-run-netns\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691218 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-cni-netd\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691242 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51abc387-da08-42d7-b10e-35d83114baf7-host-slash\") pod \"iptables-alerter-bjf2s\" (UID: \"51abc387-da08-42d7-b10e-35d83114baf7\") " pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-hostroot\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691286 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-run-openvswitch\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691308 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-run-ovn\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691331 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-run-k8s-cni-cncf-io\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691380 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-run-systemd\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691404 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-etc-openvswitch\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691452 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-os-release\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691477 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv2ls\" (UniqueName: \"kubernetes.io/projected/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-kube-api-access-rv2ls\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691501 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-socket-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691523 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vzcx\" (UniqueName: \"kubernetes.io/projected/86c772bf-658d-41c4-81e4-f4642d3e47d6-kube-api-access-6vzcx\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691546 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-systemd\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691568 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-lib-modules\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691597 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-conf-dir\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691625 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lf4h\" (UniqueName: \"kubernetes.io/projected/d5979744-4da4-40e1-9001-477372c8d2c0-kube-api-access-6lf4h\") pod \"node-resolver-xp92g\" (UID: \"d5979744-4da4-40e1-9001-477372c8d2c0\") " pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691647 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-run-netns\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691669 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-slash\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691730 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-sysctl-d\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691772 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-sys\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-sysctl-conf\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691827 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-host\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691853 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-cni-dir\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-var-lib-cni-multus\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691906 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-modprobe-d\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.691963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691929 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-tuned\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691952 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-system-cni-dir\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691975 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-socket-dir-parent\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.691999 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-var-lib-cni-bin\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692189 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-etc-kubernetes\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692256 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-registration-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692286 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-node-log\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692359 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-tmp\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692498 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/51abc387-da08-42d7-b10e-35d83114baf7-iptables-alerter-script\") pod \"iptables-alerter-bjf2s\" (UID: \"51abc387-da08-42d7-b10e-35d83114baf7\") " pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692527 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-cnibin\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692542 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-run-multus-certs\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692556 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13529312-4c7d-4dfe-bebb-61522388b930-agent-certs\") pod \"konnectivity-agent-9z8pf\" (UID: \"13529312-4c7d-4dfe-bebb-61522388b930\") " pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692605 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-log-socket\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692654 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-daemon-config\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.692713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692690 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692716 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13529312-4c7d-4dfe-bebb-61522388b930-konnectivity-ca\") pod \"konnectivity-agent-9z8pf\" (UID: \"13529312-4c7d-4dfe-bebb-61522388b930\") " pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692779 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-sysconfig\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-run\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692856 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqwjn\" (UniqueName: \"kubernetes.io/projected/818c75c1-e3d8-4a90-89de-ade8164c6d4a-kube-api-access-nqwjn\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692874 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86c772bf-658d-41c4-81e4-f4642d3e47d6-env-overrides\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692891 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86c772bf-658d-41c4-81e4-f4642d3e47d6-ovn-node-metrics-cert\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692934 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7l6s\" (UniqueName: \"kubernetes.io/projected/51abc387-da08-42d7-b10e-35d83114baf7-kube-api-access-q7l6s\") pod \"iptables-alerter-bjf2s\" (UID: \"51abc387-da08-42d7-b10e-35d83114baf7\") " pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.692980 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-device-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.693500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.693011 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-sys-fs\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.715961 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.715940 2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 20:34:01.717494 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.717462 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-31 20:29:00 +0000 UTC" deadline="2028-01-23 20:55:44.729086936 +0000 UTC" Jun 01 20:34:01.717494 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.717492 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14424h21m43.011598158s" Jun 01 20:34:01.781567 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.781078 2559 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 01 20:34:01.793983 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.793952 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86c772bf-658d-41c4-81e4-f4642d3e47d6-ovn-node-metrics-cert\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.794151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.793988 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q7l6s\" (UniqueName: \"kubernetes.io/projected/51abc387-da08-42d7-b10e-35d83114baf7-kube-api-access-q7l6s\") pod \"iptables-alerter-bjf2s\" (UID: \"51abc387-da08-42d7-b10e-35d83114baf7\") " pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.794151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794014 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-device-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.794151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794038 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-sys-fs\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.794151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794058 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9601c936-7a8f-401f-80b3-0e4a53bd248a-cni-binary-copy\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.794151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794084 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.794151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794113 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-kubelet\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.794151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794116 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-device-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.794151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794137 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-kubernetes\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794165 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm95z\" (UniqueName: \"kubernetes.io/projected/c0d1f181-3281-404c-a019-c7a99930d33d-kube-api-access-cm95z\") pod \"node-ca-cx8lv\" (UID: \"c0d1f181-3281-404c-a019-c7a99930d33d\") " pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794191 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-system-cni-dir\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794209 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-sys-fs\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794230 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-os-release\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794256 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794268 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-kubernetes\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794279 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86c772bf-658d-41c4-81e4-f4642d3e47d6-ovnkube-script-lib\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794320 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-cni-bin\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794351 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-var-lib-kubelet\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794380 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-92m8x\" (UniqueName: \"kubernetes.io/projected/d3172bfd-9aca-4acd-adf9-40803933a7eb-kube-api-access-92m8x\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794447 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794459 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5979744-4da4-40e1-9001-477372c8d2c0-hosts-file\") pod \"node-resolver-xp92g\" (UID: \"d5979744-4da4-40e1-9001-477372c8d2c0\") " pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794511 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-var-lib-openvswitch\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794521 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-cni-bin\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.794574 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794543 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-kubelet\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:01.794621 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794661 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86c772bf-658d-41c4-81e4-f4642d3e47d6-ovnkube-config\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794692 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-var-lib-openvswitch\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794714 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5979744-4da4-40e1-9001-477372c8d2c0-hosts-file\") pod \"node-resolver-xp92g\" (UID: \"d5979744-4da4-40e1-9001-477372c8d2c0\") " pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794748 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-var-lib-kubelet\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794757 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-cni-binary-copy\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:01.794828 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs podName:818c75c1-e3d8-4a90-89de-ade8164c6d4a nodeName:}" failed. No retries permitted until 2026-06-01 20:34:02.294805023 +0000 UTC m=+3.002177554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs") pod "network-metrics-daemon-z6xn6" (UID: "818c75c1-e3d8-4a90-89de-ade8164c6d4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9601c936-7a8f-401f-80b3-0e4a53bd248a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.794951 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-systemd-units\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795017 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-systemd-units\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795029 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qk87c\" (UniqueName: \"kubernetes.io/projected/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-kube-api-access-qk87c\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795057 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-var-lib-kubelet\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795083 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795099 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86c772bf-658d-41c4-81e4-f4642d3e47d6-ovnkube-script-lib\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795108 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5979744-4da4-40e1-9001-477372c8d2c0-tmp-dir\") pod \"node-resolver-xp92g\" (UID: \"d5979744-4da4-40e1-9001-477372c8d2c0\") " pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795134 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:01.795223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795159 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-run-netns\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795186 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-cni-netd\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795199 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795216 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51abc387-da08-42d7-b10e-35d83114baf7-host-slash\") pod \"iptables-alerter-bjf2s\" (UID: \"51abc387-da08-42d7-b10e-35d83114baf7\") " pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795221 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86c772bf-658d-41c4-81e4-f4642d3e47d6-ovnkube-config\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795240 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-hostroot\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795266 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-run-openvswitch\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795270 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-run-netns\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795273 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-cni-netd\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795291 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-run-ovn\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795317 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-hostroot\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795318 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795354 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-cni-binary-copy\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795360 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795242 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-var-lib-kubelet\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795384 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5979744-4da4-40e1-9001-477372c8d2c0-tmp-dir\") pod \"node-resolver-xp92g\" (UID: \"d5979744-4da4-40e1-9001-477372c8d2c0\") " pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795385 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-run-k8s-cni-cncf-io\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795438 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-run-openvswitch\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795445 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51abc387-da08-42d7-b10e-35d83114baf7-host-slash\") pod \"iptables-alerter-bjf2s\" (UID: \"51abc387-da08-42d7-b10e-35d83114baf7\") " pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795450 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-run-k8s-cni-cncf-io\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795451 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkst9\" (UniqueName: \"kubernetes.io/projected/9601c936-7a8f-401f-80b3-0e4a53bd248a-kube-api-access-fkst9\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795492 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-run-ovn\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795503 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-run-systemd\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795532 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-etc-openvswitch\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795558 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-os-release\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795563 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-run-systemd\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795592 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rv2ls\" (UniqueName: \"kubernetes.io/projected/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-kube-api-access-rv2ls\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795607 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-etc-openvswitch\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795616 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-socket-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795635 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-os-release\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795651 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6vzcx\" (UniqueName: \"kubernetes.io/projected/86c772bf-658d-41c4-81e4-f4642d3e47d6-kube-api-access-6vzcx\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795677 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-systemd\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795708 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-lib-modules\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795712 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-socket-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795735 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-conf-dir\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.796835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795766 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0d1f181-3281-404c-a019-c7a99930d33d-host\") pod \"node-ca-cx8lv\" (UID: \"c0d1f181-3281-404c-a019-c7a99930d33d\") " pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795795 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lf4h\" (UniqueName: \"kubernetes.io/projected/d5979744-4da4-40e1-9001-477372c8d2c0-kube-api-access-6lf4h\") pod \"node-resolver-xp92g\" (UID: \"d5979744-4da4-40e1-9001-477372c8d2c0\") " pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795807 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-conf-dir\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795819 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-run-netns\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795766 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-systemd\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795809 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-lib-modules\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795852 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-run-netns\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795952 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-slash\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.795979 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-sysctl-d\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796001 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-sys\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796025 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-sysctl-conf\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796030 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-slash\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796048 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-host\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796072 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-cni-dir\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796079 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-sys\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796101 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-var-lib-cni-multus\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796122 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-host\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796125 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-sysctl-d\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.797611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796146 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-var-lib-cni-multus\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796169 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-cni-dir\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796195 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0d1f181-3281-404c-a019-c7a99930d33d-serviceca\") pod \"node-ca-cx8lv\" (UID: \"c0d1f181-3281-404c-a019-c7a99930d33d\") " pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796231 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-sysctl-conf\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796232 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-cnibin\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796271 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-modprobe-d\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796289 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-tuned\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796308 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-system-cni-dir\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796333 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-socket-dir-parent\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796358 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-var-lib-cni-bin\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796372 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-modprobe-d\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796382 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-etc-kubernetes\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796415 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-etc-kubernetes\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796431 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-registration-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796487 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-registration-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796479 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-system-cni-dir\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796507 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-node-log\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796533 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-tmp\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.798471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796548 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-socket-dir-parent\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796558 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/51abc387-da08-42d7-b10e-35d83114baf7-iptables-alerter-script\") pod \"iptables-alerter-bjf2s\" (UID: \"51abc387-da08-42d7-b10e-35d83114baf7\") " pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796584 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-cnibin\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796609 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-run-multus-certs\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796612 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-var-lib-cni-bin\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796637 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9601c936-7a8f-401f-80b3-0e4a53bd248a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796584 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-node-log\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796664 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-cnibin\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796668 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13529312-4c7d-4dfe-bebb-61522388b930-agent-certs\") pod \"konnectivity-agent-9z8pf\" (UID: \"13529312-4c7d-4dfe-bebb-61522388b930\") " pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796731 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-log-socket\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796758 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-daemon-config\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796768 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-host-run-multus-certs\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796782 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796810 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796784 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-log-socket\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796835 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13529312-4c7d-4dfe-bebb-61522388b930-konnectivity-ca\") pod \"konnectivity-agent-9z8pf\" (UID: \"13529312-4c7d-4dfe-bebb-61522388b930\") " pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796890 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-sysconfig\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.799129 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796917 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-run\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796945 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nqwjn\" (UniqueName: \"kubernetes.io/projected/818c75c1-e3d8-4a90-89de-ade8164c6d4a-kube-api-access-nqwjn\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.796994 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-run\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.797025 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86c772bf-658d-41c4-81e4-f4642d3e47d6-env-overrides\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.797128 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3172bfd-9aca-4acd-adf9-40803933a7eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.797177 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86c772bf-658d-41c4-81e4-f4642d3e47d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.797336 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13529312-4c7d-4dfe-bebb-61522388b930-konnectivity-ca\") pod \"konnectivity-agent-9z8pf\" (UID: \"13529312-4c7d-4dfe-bebb-61522388b930\") " pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.797445 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86c772bf-658d-41c4-81e4-f4642d3e47d6-env-overrides\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.797474 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-sysconfig\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.797861 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/51abc387-da08-42d7-b10e-35d83114baf7-iptables-alerter-script\") pod \"iptables-alerter-bjf2s\" (UID: \"51abc387-da08-42d7-b10e-35d83114baf7\") " pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.798123 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86c772bf-658d-41c4-81e4-f4642d3e47d6-ovn-node-metrics-cert\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.798734 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-multus-daemon-config\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.799293 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-etc-tuned\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.799787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.799485 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-tmp\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.800357 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.799967 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13529312-4c7d-4dfe-bebb-61522388b930-agent-certs\") pod \"konnectivity-agent-9z8pf\" (UID: \"13529312-4c7d-4dfe-bebb-61522388b930\") " pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:01.801036 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:01.800867 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 20:34:01.801036 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:01.800891 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 20:34:01.801036 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:01.800903 2559 projected.go:194] Error preparing data for projected volume kube-api-access-lvblj for pod openshift-network-diagnostics/network-check-target-p6w46: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:01.801036 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:01.800966 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj podName:20363d2d-83dd-498f-9629-231a4496db4c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:02.300949133 +0000 UTC m=+3.008321666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lvblj" (UniqueName: "kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj") pod "network-check-target-p6w46" (UID: "20363d2d-83dd-498f-9629-231a4496db4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:01.802279 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.802254 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7l6s\" (UniqueName: \"kubernetes.io/projected/51abc387-da08-42d7-b10e-35d83114baf7-kube-api-access-q7l6s\") pod \"iptables-alerter-bjf2s\" (UID: \"51abc387-da08-42d7-b10e-35d83114baf7\") " pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:01.802509 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.802486 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92m8x\" (UniqueName: \"kubernetes.io/projected/d3172bfd-9aca-4acd-adf9-40803933a7eb-kube-api-access-92m8x\") pod \"aws-ebs-csi-driver-node-778tz\" (UID: \"d3172bfd-9aca-4acd-adf9-40803933a7eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:01.802897 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.802875 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk87c\" (UniqueName: \"kubernetes.io/projected/d8dfda38-a9d4-4fdc-a762-f94160c3ee59-kube-api-access-qk87c\") pod \"tuned-rw9bx\" (UID: \"d8dfda38-a9d4-4fdc-a762-f94160c3ee59\") " pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:01.803757 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.803737 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lf4h\" (UniqueName: \"kubernetes.io/projected/d5979744-4da4-40e1-9001-477372c8d2c0-kube-api-access-6lf4h\") pod \"node-resolver-xp92g\" (UID: \"d5979744-4da4-40e1-9001-477372c8d2c0\") " pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:01.804004 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.803959 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv2ls\" (UniqueName: \"kubernetes.io/projected/ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9-kube-api-access-rv2ls\") pod \"multus-q95l8\" (UID: \"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9\") " pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.804828 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.804807 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqwjn\" (UniqueName: \"kubernetes.io/projected/818c75c1-e3d8-4a90-89de-ade8164c6d4a-kube-api-access-nqwjn\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:01.804901 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.804837 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vzcx\" (UniqueName: \"kubernetes.io/projected/86c772bf-658d-41c4-81e4-f4642d3e47d6-kube-api-access-6vzcx\") pod \"ovnkube-node-knvtx\" (UID: \"86c772bf-658d-41c4-81e4-f4642d3e47d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.898209 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898155 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0d1f181-3281-404c-a019-c7a99930d33d-host\") pod \"node-ca-cx8lv\" (UID: \"c0d1f181-3281-404c-a019-c7a99930d33d\") " pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.898209 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898213 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0d1f181-3281-404c-a019-c7a99930d33d-serviceca\") pod \"node-ca-cx8lv\" (UID: \"c0d1f181-3281-404c-a019-c7a99930d33d\") " pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898231 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-cnibin\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898259 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9601c936-7a8f-401f-80b3-0e4a53bd248a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898262 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0d1f181-3281-404c-a019-c7a99930d33d-host\") pod \"node-ca-cx8lv\" (UID: \"c0d1f181-3281-404c-a019-c7a99930d33d\") " pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898294 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9601c936-7a8f-401f-80b3-0e4a53bd248a-cni-binary-copy\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898314 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898319 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-cnibin\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898340 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cm95z\" (UniqueName: \"kubernetes.io/projected/c0d1f181-3281-404c-a019-c7a99930d33d-kube-api-access-cm95z\") pod \"node-ca-cx8lv\" (UID: \"c0d1f181-3281-404c-a019-c7a99930d33d\") " pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898365 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-system-cni-dir\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898389 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-os-release\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.898446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898447 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9601c936-7a8f-401f-80b3-0e4a53bd248a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.899060 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898504 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fkst9\" (UniqueName: \"kubernetes.io/projected/9601c936-7a8f-401f-80b3-0e4a53bd248a-kube-api-access-fkst9\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.899060 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898605 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-system-cni-dir\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.899060 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898657 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-os-release\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.899060 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898707 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0d1f181-3281-404c-a019-c7a99930d33d-serviceca\") pod \"node-ca-cx8lv\" (UID: \"c0d1f181-3281-404c-a019-c7a99930d33d\") " pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.899060 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898732 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9601c936-7a8f-401f-80b3-0e4a53bd248a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.899060 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.898851 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9601c936-7a8f-401f-80b3-0e4a53bd248a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.899060 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.899056 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9601c936-7a8f-401f-80b3-0e4a53bd248a-cni-binary-copy\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.899397 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.899111 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9601c936-7a8f-401f-80b3-0e4a53bd248a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.906032 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.906006 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm95z\" (UniqueName: \"kubernetes.io/projected/c0d1f181-3281-404c-a019-c7a99930d33d-kube-api-access-cm95z\") pod \"node-ca-cx8lv\" (UID: \"c0d1f181-3281-404c-a019-c7a99930d33d\") " pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:01.906443 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.906401 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkst9\" (UniqueName: \"kubernetes.io/projected/9601c936-7a8f-401f-80b3-0e4a53bd248a-kube-api-access-fkst9\") pod \"multus-additional-cni-plugins-pd28x\" (UID: \"9601c936-7a8f-401f-80b3-0e4a53bd248a\") " pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:01.981492 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.981405 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q95l8" Jun 01 20:34:01.986210 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.986189 2559 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 20:34:01.989776 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.989758 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:01.997101 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:01.997080 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:02.001766 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.001746 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" Jun 01 20:34:02.009326 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.009300 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bjf2s" Jun 01 20:34:02.014955 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.014932 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" Jun 01 20:34:02.021591 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.021568 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xp92g" Jun 01 20:34:02.030099 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.030078 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cx8lv" Jun 01 20:34:02.034748 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.034727 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pd28x" Jun 01 20:34:02.300948 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.300862 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:02.301111 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:02.301033 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:02.301111 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:02.301097 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs podName:818c75c1-e3d8-4a90-89de-ade8164c6d4a nodeName:}" failed. No retries permitted until 2026-06-01 20:34:03.301080636 +0000 UTC m=+4.008453155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs") pod "network-metrics-daemon-z6xn6" (UID: "818c75c1-e3d8-4a90-89de-ade8164c6d4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:02.359726 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:02.359498 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13529312_4c7d_4dfe_bebb_61522388b930.slice/crio-6a1766bf235d622d07710a338ee4f8b19287b505c6001949874fe09526f6fa96 WatchSource:0}: Error finding container 6a1766bf235d622d07710a338ee4f8b19287b505c6001949874fe09526f6fa96: Status 404 returned error can't find the container with id 6a1766bf235d622d07710a338ee4f8b19287b505c6001949874fe09526f6fa96 Jun 01 20:34:02.360979 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:02.360896 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8dfda38_a9d4_4fdc_a762_f94160c3ee59.slice/crio-147a220b775b9984670eb3a656287f7b214b35ca716ebf5f29267e0398796cea WatchSource:0}: Error finding container 147a220b775b9984670eb3a656287f7b214b35ca716ebf5f29267e0398796cea: Status 404 returned error can't find the container with id 147a220b775b9984670eb3a656287f7b214b35ca716ebf5f29267e0398796cea Jun 01 20:34:02.361599 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:02.361575 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5979744_4da4_40e1_9001_477372c8d2c0.slice/crio-222b75b300736a97d028cc0c68c7de7560e0c079e7f52ab52ed6c582330af74d WatchSource:0}: Error finding container 222b75b300736a97d028cc0c68c7de7560e0c079e7f52ab52ed6c582330af74d: Status 404 returned error can't find the container with id 222b75b300736a97d028cc0c68c7de7560e0c079e7f52ab52ed6c582330af74d Jun 01 20:34:02.362742 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:02.362626 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d1f181_3281_404c_a019_c7a99930d33d.slice/crio-672cbcfd38b4bbf7dc3a15d75f3c9812cdf8ea39208d8d5b7406c0027e0ff3ad WatchSource:0}: Error finding container 672cbcfd38b4bbf7dc3a15d75f3c9812cdf8ea39208d8d5b7406c0027e0ff3ad: Status 404 returned error can't find the container with id 672cbcfd38b4bbf7dc3a15d75f3c9812cdf8ea39208d8d5b7406c0027e0ff3ad Jun 01 20:34:02.364821 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:02.364798 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9601c936_7a8f_401f_80b3_0e4a53bd248a.slice/crio-3626ac3772aaa2f17d4ee54aaa6193656b6be7ebe9f92fc75c28843a88e76522 WatchSource:0}: Error finding container 3626ac3772aaa2f17d4ee54aaa6193656b6be7ebe9f92fc75c28843a88e76522: Status 404 returned error can't find the container with id 3626ac3772aaa2f17d4ee54aaa6193656b6be7ebe9f92fc75c28843a88e76522 Jun 01 20:34:02.365841 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:02.365739 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51abc387_da08_42d7_b10e_35d83114baf7.slice/crio-8d917f150a6bf1509e93bfedea459fcde821e8460c99af4be47334914c588c04 WatchSource:0}: Error finding container 8d917f150a6bf1509e93bfedea459fcde821e8460c99af4be47334914c588c04: Status 404 returned error can't find the container with id 8d917f150a6bf1509e93bfedea459fcde821e8460c99af4be47334914c588c04 Jun 01 20:34:02.367122 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:02.366970 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c772bf_658d_41c4_81e4_f4642d3e47d6.slice/crio-7c6a0df0e6e3396ed521ccc4329156011ea330aa84518f7c5432dc33181e7225 WatchSource:0}: Error finding container 7c6a0df0e6e3396ed521ccc4329156011ea330aa84518f7c5432dc33181e7225: Status 404 returned error can't find the container with id 7c6a0df0e6e3396ed521ccc4329156011ea330aa84518f7c5432dc33181e7225 Jun 01 20:34:02.368411 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:02.368388 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4d0b21_4002_4ec4_8bc4_5e4a7e3225d9.slice/crio-c53dc40876f7e3c9a4e22430b179fdbed460591e80e5efaa1233b8a93827135b WatchSource:0}: Error finding container c53dc40876f7e3c9a4e22430b179fdbed460591e80e5efaa1233b8a93827135b: Status 404 returned error can't find the container with id c53dc40876f7e3c9a4e22430b179fdbed460591e80e5efaa1233b8a93827135b Jun 01 20:34:02.369902 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:02.369882 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3172bfd_9aca_4acd_adf9_40803933a7eb.slice/crio-c130ba1f2d360d8cc8b5ec64af7a1db9621438b23ec16de49a4707bd5cc1f8fa WatchSource:0}: Error finding container c130ba1f2d360d8cc8b5ec64af7a1db9621438b23ec16de49a4707bd5cc1f8fa: Status 404 returned error can't find the container with id c130ba1f2d360d8cc8b5ec64af7a1db9621438b23ec16de49a4707bd5cc1f8fa Jun 01 20:34:02.401259 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.401232 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:02.401651 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:02.401587 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 20:34:02.401651 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:02.401614 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 20:34:02.401651 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:02.401643 2559 projected.go:194] Error preparing data for projected volume kube-api-access-lvblj for pod openshift-network-diagnostics/network-check-target-p6w46: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:02.401834 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:02.401715 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj podName:20363d2d-83dd-498f-9629-231a4496db4c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:03.401685105 +0000 UTC m=+4.109057639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvblj" (UniqueName: "kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj") pod "network-check-target-p6w46" (UID: "20363d2d-83dd-498f-9629-231a4496db4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:02.718225 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.718183 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-31 20:29:00 +0000 UTC" deadline="2028-01-28 05:59:08.725371027 +0000 UTC" Jun 01 20:34:02.718225 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.718222 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14529h25m6.007153049s" Jun 01 20:34:02.861146 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.861060 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9z8pf" event={"ID":"13529312-4c7d-4dfe-bebb-61522388b930","Type":"ContainerStarted","Data":"6a1766bf235d622d07710a338ee4f8b19287b505c6001949874fe09526f6fa96"} Jun 01 20:34:02.882492 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.881696 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal" event={"ID":"b297c8edefffe884478fc112206a107d","Type":"ContainerStarted","Data":"8138274a770c3a2ebfa2128c23b71905f70d8d8d459e580688744a6cdc59d311"} Jun 01 20:34:02.894486 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.893278 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-183.ec2.internal" podStartSLOduration=2.893262844 podStartE2EDuration="2.893262844s" podCreationTimestamp="2026-06-01 20:34:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-01 20:34:02.893050839 +0000 UTC m=+3.600423378" watchObservedRunningTime="2026-06-01 20:34:02.893262844 +0000 UTC m=+3.600635381" Jun 01 20:34:02.894486 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.894411 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" event={"ID":"86c772bf-658d-41c4-81e4-f4642d3e47d6","Type":"ContainerStarted","Data":"7c6a0df0e6e3396ed521ccc4329156011ea330aa84518f7c5432dc33181e7225"} Jun 01 20:34:02.902073 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.902007 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerStarted","Data":"3626ac3772aaa2f17d4ee54aaa6193656b6be7ebe9f92fc75c28843a88e76522"} Jun 01 20:34:02.910686 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.910657 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cx8lv" event={"ID":"c0d1f181-3281-404c-a019-c7a99930d33d","Type":"ContainerStarted","Data":"672cbcfd38b4bbf7dc3a15d75f3c9812cdf8ea39208d8d5b7406c0027e0ff3ad"} Jun 01 20:34:02.920204 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.920177 2559 generic.go:358] "Generic (PLEG): container finished" podID="ab6fe5603972050dfc53ab2a7e03a2b3" containerID="8362e7eed341d79b25bd396d0babbed6c602702a3d78b04ebbf34164ddf0f688" exitCode=0 Jun 01 20:34:02.920321 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.920269 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" event={"ID":"ab6fe5603972050dfc53ab2a7e03a2b3","Type":"ContainerDied","Data":"8362e7eed341d79b25bd396d0babbed6c602702a3d78b04ebbf34164ddf0f688"} Jun 01 20:34:02.935939 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.935905 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" event={"ID":"d3172bfd-9aca-4acd-adf9-40803933a7eb","Type":"ContainerStarted","Data":"c130ba1f2d360d8cc8b5ec64af7a1db9621438b23ec16de49a4707bd5cc1f8fa"} Jun 01 20:34:02.952797 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.952598 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q95l8" event={"ID":"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9","Type":"ContainerStarted","Data":"c53dc40876f7e3c9a4e22430b179fdbed460591e80e5efaa1233b8a93827135b"} Jun 01 20:34:02.975504 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.975379 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bjf2s" event={"ID":"51abc387-da08-42d7-b10e-35d83114baf7","Type":"ContainerStarted","Data":"8d917f150a6bf1509e93bfedea459fcde821e8460c99af4be47334914c588c04"} Jun 01 20:34:02.986771 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.986617 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xp92g" event={"ID":"d5979744-4da4-40e1-9001-477372c8d2c0","Type":"ContainerStarted","Data":"222b75b300736a97d028cc0c68c7de7560e0c079e7f52ab52ed6c582330af74d"} Jun 01 20:34:02.988715 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:02.988597 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" event={"ID":"d8dfda38-a9d4-4fdc-a762-f94160c3ee59","Type":"ContainerStarted","Data":"147a220b775b9984670eb3a656287f7b214b35ca716ebf5f29267e0398796cea"} Jun 01 20:34:03.307770 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:03.307166 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:03.307770 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:03.307324 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:03.307770 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:03.307398 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs podName:818c75c1-e3d8-4a90-89de-ade8164c6d4a nodeName:}" failed. No retries permitted until 2026-06-01 20:34:05.307378658 +0000 UTC m=+6.014751175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs") pod "network-metrics-daemon-z6xn6" (UID: "818c75c1-e3d8-4a90-89de-ade8164c6d4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:03.408369 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:03.408260 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:03.408598 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:03.408439 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 20:34:03.408598 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:03.408459 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 20:34:03.408598 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:03.408471 2559 projected.go:194] Error preparing data for projected volume kube-api-access-lvblj for pod openshift-network-diagnostics/network-check-target-p6w46: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:03.408598 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:03.408536 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj podName:20363d2d-83dd-498f-9629-231a4496db4c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:05.408515536 +0000 UTC m=+6.115888051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvblj" (UniqueName: "kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj") pod "network-check-target-p6w46" (UID: "20363d2d-83dd-498f-9629-231a4496db4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:03.828972 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:03.828891 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:03.829522 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:03.829021 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:03.829522 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:03.829459 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:03.829636 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:03.829563 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:04.010134 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:04.010097 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" event={"ID":"ab6fe5603972050dfc53ab2a7e03a2b3","Type":"ContainerStarted","Data":"a6b1a0f57243121795e8710e0bb52e0fc414d67bff5ddd69f4f9b286b69a5a33"} Jun 01 20:34:05.323885 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:05.323409 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:05.323885 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:05.323577 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:05.323885 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:05.323662 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs podName:818c75c1-e3d8-4a90-89de-ade8164c6d4a nodeName:}" failed. No retries permitted until 2026-06-01 20:34:09.323640248 +0000 UTC m=+10.031012783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs") pod "network-metrics-daemon-z6xn6" (UID: "818c75c1-e3d8-4a90-89de-ade8164c6d4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:05.424159 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:05.424119 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:05.424354 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:05.424287 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 20:34:05.424354 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:05.424306 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 20:34:05.424354 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:05.424318 2559 projected.go:194] Error preparing data for projected volume kube-api-access-lvblj for pod openshift-network-diagnostics/network-check-target-p6w46: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:05.424577 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:05.424377 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj podName:20363d2d-83dd-498f-9629-231a4496db4c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:09.424358976 +0000 UTC m=+10.131731496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvblj" (UniqueName: "kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj") pod "network-check-target-p6w46" (UID: "20363d2d-83dd-498f-9629-231a4496db4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:05.827553 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:05.827526 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:05.827722 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:05.827638 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:05.827799 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:05.827734 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:05.827914 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:05.827868 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:07.827153 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:07.827106 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:07.827625 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:07.827239 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:07.827625 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:07.827124 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:07.827750 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:07.827687 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:09.353226 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:09.353187 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:09.353709 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:09.353466 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:09.353709 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:09.353570 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs podName:818c75c1-e3d8-4a90-89de-ade8164c6d4a nodeName:}" failed. No retries permitted until 2026-06-01 20:34:17.3535473 +0000 UTC m=+18.060919828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs") pod "network-metrics-daemon-z6xn6" (UID: "818c75c1-e3d8-4a90-89de-ade8164c6d4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:09.454022 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:09.453983 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:09.454254 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:09.454157 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 20:34:09.454254 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:09.454185 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 20:34:09.454254 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:09.454197 2559 projected.go:194] Error preparing data for projected volume kube-api-access-lvblj for pod openshift-network-diagnostics/network-check-target-p6w46: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:09.454416 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:09.454262 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj podName:20363d2d-83dd-498f-9629-231a4496db4c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:17.454243602 +0000 UTC m=+18.161616130 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvblj" (UniqueName: "kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj") pod "network-check-target-p6w46" (UID: "20363d2d-83dd-498f-9629-231a4496db4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:09.827168 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:09.827142 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:09.827293 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:09.827225 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:09.827369 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:09.827342 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:09.827532 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:09.827508 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:11.826803 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:11.826771 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:11.826803 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:11.826804 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:11.827310 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:11.826893 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:11.827310 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:11.827033 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:12.506799 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.506749 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-183.ec2.internal" podStartSLOduration=12.506733117 podStartE2EDuration="12.506733117s" podCreationTimestamp="2026-06-01 20:34:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-01 20:34:04.02279299 +0000 UTC m=+4.730165529" watchObservedRunningTime="2026-06-01 20:34:12.506733117 +0000 UTC m=+13.214105653" Jun 01 20:34:12.506968 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.506942 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-p8f5k"] Jun 01 20:34:12.521752 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.521710 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:12.521930 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:12.521802 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:12.578341 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.578245 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/752796d9-4fc8-4dd8-b91c-611bdc542833-dbus\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:12.578341 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.578291 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/752796d9-4fc8-4dd8-b91c-611bdc542833-kubelet-config\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:12.578569 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.578380 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:12.679662 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.679618 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:12.679852 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.679683 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/752796d9-4fc8-4dd8-b91c-611bdc542833-dbus\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:12.679852 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.679710 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/752796d9-4fc8-4dd8-b91c-611bdc542833-kubelet-config\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:12.679852 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.679815 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/752796d9-4fc8-4dd8-b91c-611bdc542833-kubelet-config\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:12.680004 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:12.679924 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:12.680004 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:12.679987 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret podName:752796d9-4fc8-4dd8-b91c-611bdc542833 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:13.179967574 +0000 UTC m=+13.887340104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret") pod "global-pull-secret-syncer-p8f5k" (UID: "752796d9-4fc8-4dd8-b91c-611bdc542833") : object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:12.680316 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:12.680283 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/752796d9-4fc8-4dd8-b91c-611bdc542833-dbus\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:13.184844 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:13.184800 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:13.185305 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:13.184954 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:13.185305 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:13.185082 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret podName:752796d9-4fc8-4dd8-b91c-611bdc542833 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:14.184997841 +0000 UTC m=+14.892370371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret") pod "global-pull-secret-syncer-p8f5k" (UID: "752796d9-4fc8-4dd8-b91c-611bdc542833") : object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:13.830057 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:13.830026 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:13.830225 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:13.830035 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:13.830225 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:13.830150 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:13.830349 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:13.830233 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:14.190767 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:14.190684 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:14.191209 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:14.190836 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:14.191209 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:14.190894 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret podName:752796d9-4fc8-4dd8-b91c-611bdc542833 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:16.190880981 +0000 UTC m=+16.898253499 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret") pod "global-pull-secret-syncer-p8f5k" (UID: "752796d9-4fc8-4dd8-b91c-611bdc542833") : object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:14.827153 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:14.827108 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:14.827325 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:14.827267 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:15.827217 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:15.827181 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:15.827700 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:15.827227 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:15.827700 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:15.827316 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:15.827700 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:15.827484 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:16.205925 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:16.205841 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:16.206071 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:16.205982 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:16.206071 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:16.206043 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret podName:752796d9-4fc8-4dd8-b91c-611bdc542833 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:20.206024372 +0000 UTC m=+20.913396898 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret") pod "global-pull-secret-syncer-p8f5k" (UID: "752796d9-4fc8-4dd8-b91c-611bdc542833") : object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:16.827216 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:16.827186 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:16.827437 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:16.827297 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:17.415328 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:17.415278 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:17.415545 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:17.415453 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:17.415545 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:17.415532 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs podName:818c75c1-e3d8-4a90-89de-ade8164c6d4a nodeName:}" failed. No retries permitted until 2026-06-01 20:34:33.415510956 +0000 UTC m=+34.122883491 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs") pod "network-metrics-daemon-z6xn6" (UID: "818c75c1-e3d8-4a90-89de-ade8164c6d4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:17.515997 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:17.515961 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:17.516149 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:17.516122 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 20:34:17.516149 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:17.516147 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 20:34:17.516233 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:17.516162 2559 projected.go:194] Error preparing data for projected volume kube-api-access-lvblj for pod openshift-network-diagnostics/network-check-target-p6w46: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:17.516233 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:17.516224 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj podName:20363d2d-83dd-498f-9629-231a4496db4c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:33.516204686 +0000 UTC m=+34.223577204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvblj" (UniqueName: "kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj") pod "network-check-target-p6w46" (UID: "20363d2d-83dd-498f-9629-231a4496db4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:17.826914 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:17.826831 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:17.827071 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:17.826836 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:17.827071 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:17.826953 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:17.827071 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:17.827029 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:18.827109 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:18.827080 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:18.827534 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:18.827193 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:19.827962 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:19.827929 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:19.828478 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:19.828039 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:19.828478 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:19.828086 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:19.828478 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:19.828204 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:20.233823 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:20.233750 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:20.233951 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:20.233862 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:20.233951 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:20.233911 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret podName:752796d9-4fc8-4dd8-b91c-611bdc542833 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:28.233898372 +0000 UTC m=+28.941270887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret") pod "global-pull-secret-syncer-p8f5k" (UID: "752796d9-4fc8-4dd8-b91c-611bdc542833") : object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:20.826699 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:20.826674 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:20.826800 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:20.826787 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:21.054934 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.054888 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" event={"ID":"d3172bfd-9aca-4acd-adf9-40803933a7eb","Type":"ContainerStarted","Data":"17ffb1f8b5c52547d1401052d77232922360726d55f21891fe1b77a80afc32ae"} Jun 01 20:34:21.056029 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.056006 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q95l8" event={"ID":"ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9","Type":"ContainerStarted","Data":"694024633c9797e80fd1147ad213a99464d3fc5b8fae8ee25602cffd965d4e4d"} Jun 01 20:34:21.057553 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.057526 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xp92g" event={"ID":"d5979744-4da4-40e1-9001-477372c8d2c0","Type":"ContainerStarted","Data":"e6c931cc4ae3a1d482d9eca3922be82a649770e5b0169318e18f510c8cd6ec12"} Jun 01 20:34:21.059389 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.059366 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" event={"ID":"d8dfda38-a9d4-4fdc-a762-f94160c3ee59","Type":"ContainerStarted","Data":"bbc03e65820cb2eb3f0917e78091b2745622755fe43e542a1d2c6ccca56e1056"} Jun 01 20:34:21.060891 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.060869 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9z8pf" event={"ID":"13529312-4c7d-4dfe-bebb-61522388b930","Type":"ContainerStarted","Data":"f3d865971b14d2ee88929cb7ea21ec58ee9920b93433f84cd9b0d3faebbb378e"} Jun 01 20:34:21.062284 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.062263 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" event={"ID":"86c772bf-658d-41c4-81e4-f4642d3e47d6","Type":"ContainerStarted","Data":"c392cfa386ef49c76db684e439951506c703061cce90ea71169ea9b094dd3674"} Jun 01 20:34:21.063598 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.063508 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerStarted","Data":"0dc0ee6ac7c12205a725c4ab60a837ea7f7e0e8cf13e0fc981a24d973137a1a7"} Jun 01 20:34:21.064638 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.064617 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cx8lv" event={"ID":"c0d1f181-3281-404c-a019-c7a99930d33d","Type":"ContainerStarted","Data":"acf8892d591ff7d5e38ad864a8c97ee7125af7721db27f6329af23e15794d91e"} Jun 01 20:34:21.070903 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.070856 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-q95l8" podStartSLOduration=3.699167875 podStartE2EDuration="22.070845291s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:34:02.370545139 +0000 UTC m=+3.077917670" lastFinishedPulling="2026-06-01 20:34:20.742222554 +0000 UTC m=+21.449595086" observedRunningTime="2026-06-01 20:34:21.069688726 +0000 UTC m=+21.777061264" watchObservedRunningTime="2026-06-01 20:34:21.070845291 +0000 UTC m=+21.778217828" Jun 01 20:34:21.081039 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.080990 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9z8pf" podStartSLOduration=8.531759542 podStartE2EDuration="22.080978291s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:34:02.361186316 +0000 UTC m=+3.068558832" lastFinishedPulling="2026-06-01 20:34:15.91040505 +0000 UTC m=+16.617777581" observedRunningTime="2026-06-01 20:34:21.080881458 +0000 UTC m=+21.788253996" watchObservedRunningTime="2026-06-01 20:34:21.080978291 +0000 UTC m=+21.788350827" Jun 01 20:34:21.095111 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.095056 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rw9bx" podStartSLOduration=3.725677113 podStartE2EDuration="22.09504538s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:34:02.36299057 +0000 UTC m=+3.070363087" lastFinishedPulling="2026-06-01 20:34:20.732358833 +0000 UTC m=+21.439731354" observedRunningTime="2026-06-01 20:34:21.094457761 +0000 UTC m=+21.801830295" watchObservedRunningTime="2026-06-01 20:34:21.09504538 +0000 UTC m=+21.802417914" Jun 01 20:34:21.106491 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.106442 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-xp92g" podStartSLOduration=3.740922537 podStartE2EDuration="22.106402426s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:34:02.364555729 +0000 UTC m=+3.071928259" lastFinishedPulling="2026-06-01 20:34:20.730035634 +0000 UTC m=+21.437408148" observedRunningTime="2026-06-01 20:34:21.105589698 +0000 UTC m=+21.812962245" watchObservedRunningTime="2026-06-01 20:34:21.106402426 +0000 UTC m=+21.813774965" Jun 01 20:34:21.120216 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.120079 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cx8lv" podStartSLOduration=3.755041801 podStartE2EDuration="22.120068265s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:34:02.365282513 +0000 UTC m=+3.072655030" lastFinishedPulling="2026-06-01 20:34:20.730308973 +0000 UTC m=+21.437681494" observedRunningTime="2026-06-01 20:34:21.11990598 +0000 UTC m=+21.827278517" watchObservedRunningTime="2026-06-01 20:34:21.120068265 +0000 UTC m=+21.827440801" Jun 01 20:34:21.826662 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.826624 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:21.826906 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.826638 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:21.826906 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:21.826758 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:21.826906 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:21.826883 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:21.979776 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:21.979752 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 01 20:34:22.068581 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.068548 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" event={"ID":"86c772bf-658d-41c4-81e4-f4642d3e47d6","Type":"ContainerStarted","Data":"823d1a125ec58e0c1ba704da9ca2ba807ff0b8e9e7e9aa439074313fa0677708"} Jun 01 20:34:22.069035 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.068585 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" event={"ID":"86c772bf-658d-41c4-81e4-f4642d3e47d6","Type":"ContainerStarted","Data":"c969b576a7fd5b11fe30581294b6072593e4bfd759a9b26af7865188c9612ae6"} Jun 01 20:34:22.069035 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.068596 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" event={"ID":"86c772bf-658d-41c4-81e4-f4642d3e47d6","Type":"ContainerStarted","Data":"00077b5ff9a61b92cbd13eda938c9e23e42b34c17041c6afaccccd72a56b91ae"} Jun 01 20:34:22.069035 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.068609 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" event={"ID":"86c772bf-658d-41c4-81e4-f4642d3e47d6","Type":"ContainerStarted","Data":"afc38e2cf2856b839a9ddb0a5e088f3b1caf141d2e5161fdb65e1448a7c08ecc"} Jun 01 20:34:22.069035 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.068622 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" event={"ID":"86c772bf-658d-41c4-81e4-f4642d3e47d6","Type":"ContainerStarted","Data":"04e9b84caff982ce84070d3999ed743ad7da420b60ede1bbdc976c16b1dd94b2"} Jun 01 20:34:22.069832 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.069813 2559 generic.go:358] "Generic (PLEG): container finished" podID="9601c936-7a8f-401f-80b3-0e4a53bd248a" containerID="0dc0ee6ac7c12205a725c4ab60a837ea7f7e0e8cf13e0fc981a24d973137a1a7" exitCode=0 Jun 01 20:34:22.069927 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.069906 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerDied","Data":"0dc0ee6ac7c12205a725c4ab60a837ea7f7e0e8cf13e0fc981a24d973137a1a7"} Jun 01 20:34:22.071494 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.071474 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" event={"ID":"d3172bfd-9aca-4acd-adf9-40803933a7eb","Type":"ContainerStarted","Data":"054cb36a1e76641673ee56be6d7956da83671632ebcf8e70402e3eeed72d5d30"} Jun 01 20:34:22.750444 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.750313 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-01T20:34:21.979773008Z","UUID":"a32af300-efeb-4e12-96af-6b33fadc2679","Handler":null,"Name":"","Endpoint":""} Jun 01 20:34:22.754292 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.754264 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 01 20:34:22.754292 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.754294 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 01 20:34:22.826895 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:22.826867 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:22.827070 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:22.826978 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:23.074882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:23.074798 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bjf2s" event={"ID":"51abc387-da08-42d7-b10e-35d83114baf7","Type":"ContainerStarted","Data":"5c112c8960f0c7ba9e876fc0c62b6e5d40a8c1e217a9b2d8e5c008d064f0c279"} Jun 01 20:34:23.076983 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:23.076954 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" event={"ID":"d3172bfd-9aca-4acd-adf9-40803933a7eb","Type":"ContainerStarted","Data":"6005a42bdf78423c39c601f787e341cc6131281ac0ac3e57a9d696dfc1c2e67c"} Jun 01 20:34:23.087872 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:23.087827 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bjf2s" podStartSLOduration=10.545034003 podStartE2EDuration="24.08781428s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:34:02.367508885 +0000 UTC m=+3.074881400" lastFinishedPulling="2026-06-01 20:34:15.910289157 +0000 UTC m=+16.617661677" observedRunningTime="2026-06-01 20:34:23.087277223 +0000 UTC m=+23.794649765" watchObservedRunningTime="2026-06-01 20:34:23.08781428 +0000 UTC m=+23.795186820" Jun 01 20:34:23.102561 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:23.102520 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-778tz" podStartSLOduration=3.628939515 podStartE2EDuration="24.102507662s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:34:02.371831912 +0000 UTC m=+3.079204430" lastFinishedPulling="2026-06-01 20:34:22.845400048 +0000 UTC m=+23.552772577" observedRunningTime="2026-06-01 20:34:23.102326703 +0000 UTC m=+23.809699241" watchObservedRunningTime="2026-06-01 20:34:23.102507662 +0000 UTC m=+23.809880198" Jun 01 20:34:23.826668 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:23.826526 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:23.826793 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:23.826537 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:23.826793 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:23.826746 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:23.826871 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:23.826840 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:24.082021 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:24.081937 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" event={"ID":"86c772bf-658d-41c4-81e4-f4642d3e47d6","Type":"ContainerStarted","Data":"c8b60720715f2ccc239766cae582a4638fd4236c92da7568f5a5faf38f16e0e9"} Jun 01 20:34:24.826714 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:24.826680 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:24.826924 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:24.826794 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:25.189490 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:25.189462 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:25.190111 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:25.190094 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:25.826955 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:25.826776 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:25.827081 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:25.826776 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:25.827081 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:25.827058 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:25.827143 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:25.827093 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:26.089708 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:26.089666 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" event={"ID":"86c772bf-658d-41c4-81e4-f4642d3e47d6","Type":"ContainerStarted","Data":"f42458679e50c04bc0e786e215651e83ef1f40f136ac2077f91798fb8e67fcee"} Jun 01 20:34:26.090370 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:26.090344 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:26.090541 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:26.090520 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9z8pf" Jun 01 20:34:26.115197 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:26.115130 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" podStartSLOduration=8.521375749 podStartE2EDuration="27.115110657s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:34:02.369229508 +0000 UTC m=+3.076602027" lastFinishedPulling="2026-06-01 20:34:20.962964412 +0000 UTC m=+21.670336935" observedRunningTime="2026-06-01 20:34:26.114858027 +0000 UTC m=+26.822230568" watchObservedRunningTime="2026-06-01 20:34:26.115110657 +0000 UTC m=+26.822483193" Jun 01 20:34:26.827222 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:26.827190 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:26.827906 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:26.827310 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:27.092605 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.092527 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:27.092605 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.092558 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:27.092605 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.092567 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:27.108704 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.108669 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:27.109102 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.109073 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:27.620948 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.620918 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p8f5k"] Jun 01 20:34:27.621104 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.621022 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:27.621140 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:27.621105 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:27.623844 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.623806 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p6w46"] Jun 01 20:34:27.623979 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.623927 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:27.624041 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:27.624020 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:27.624406 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.624386 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z6xn6"] Jun 01 20:34:27.624535 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:27.624514 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:27.624637 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:27.624608 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:28.095260 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:28.095220 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerStarted","Data":"18b194665756cf1db8e204600cb6128843d5ac26b0d7bc13770afe74c9286df6"} Jun 01 20:34:28.294371 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:28.294337 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:28.294517 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:28.294462 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:28.294517 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:28.294514 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret podName:752796d9-4fc8-4dd8-b91c-611bdc542833 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:44.294499633 +0000 UTC m=+45.001872148 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret") pod "global-pull-secret-syncer-p8f5k" (UID: "752796d9-4fc8-4dd8-b91c-611bdc542833") : object "kube-system"/"original-pull-secret" not registered Jun 01 20:34:28.827432 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:28.827398 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:28.827594 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:28.827522 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:29.098787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:29.098706 2559 generic.go:358] "Generic (PLEG): container finished" podID="9601c936-7a8f-401f-80b3-0e4a53bd248a" containerID="18b194665756cf1db8e204600cb6128843d5ac26b0d7bc13770afe74c9286df6" exitCode=0 Jun 01 20:34:29.099128 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:29.098790 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerDied","Data":"18b194665756cf1db8e204600cb6128843d5ac26b0d7bc13770afe74c9286df6"} Jun 01 20:34:29.829814 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:29.828718 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:29.829814 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:29.828855 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:29.829814 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:29.828930 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:29.829814 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:29.829054 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:30.826969 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:30.826941 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:30.827590 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:30.827033 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:31.103529 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:31.103444 2559 generic.go:358] "Generic (PLEG): container finished" podID="9601c936-7a8f-401f-80b3-0e4a53bd248a" containerID="ab442c4e4dee1af3dba787c4ad1d081f7e99a19a7d5626ada4664d3503285ec9" exitCode=0 Jun 01 20:34:31.103529 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:31.103509 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerDied","Data":"ab442c4e4dee1af3dba787c4ad1d081f7e99a19a7d5626ada4664d3503285ec9"} Jun 01 20:34:31.827118 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:31.827088 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:31.827622 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:31.827220 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p8f5k" podUID="752796d9-4fc8-4dd8-b91c-611bdc542833" Jun 01 20:34:31.827622 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:31.827091 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:31.827622 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:31.827406 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:34:32.827174 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:32.827141 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:32.827759 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:32.827240 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p6w46" podUID="20363d2d-83dd-498f-9629-231a4496db4c" Jun 01 20:34:33.108990 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.108905 2559 generic.go:358] "Generic (PLEG): container finished" podID="9601c936-7a8f-401f-80b3-0e4a53bd248a" containerID="d65c4d5fc64bac9683bdc5099039e1a45c44171fa6bd5f7fdb28da43185f826b" exitCode=0 Jun 01 20:34:33.108990 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.108948 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerDied","Data":"d65c4d5fc64bac9683bdc5099039e1a45c44171fa6bd5f7fdb28da43185f826b"} Jun 01 20:34:33.437416 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.437388 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:33.437605 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.437554 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:33.437667 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.437624 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs podName:818c75c1-e3d8-4a90-89de-ade8164c6d4a nodeName:}" failed. No retries permitted until 2026-06-01 20:35:05.437604221 +0000 UTC m=+66.144976748 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs") pod "network-metrics-daemon-z6xn6" (UID: "818c75c1-e3d8-4a90-89de-ade8164c6d4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 20:34:33.538182 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.538147 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:33.538330 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.538298 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 20:34:33.538330 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.538316 2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 20:34:33.538330 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.538326 2559 projected.go:194] Error preparing data for projected volume kube-api-access-lvblj for pod openshift-network-diagnostics/network-check-target-p6w46: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:33.538457 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.538375 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj podName:20363d2d-83dd-498f-9629-231a4496db4c nodeName:}" failed. No retries permitted until 2026-06-01 20:35:05.53835827 +0000 UTC m=+66.245730801 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvblj" (UniqueName: "kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj") pod "network-check-target-p6w46" (UID: "20363d2d-83dd-498f-9629-231a4496db4c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 20:34:33.618146 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.618110 2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-183.ec2.internal" event="NodeReady" Jun 01 20:34:33.618311 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.618269 2559 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 01 20:34:33.659745 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.659707 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qf2m5"] Jun 01 20:34:33.662958 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.662937 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v8zqc"] Jun 01 20:34:33.663118 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.663096 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.665000 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.664974 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 01 20:34:33.665091 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.665007 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6rq65\"" Jun 01 20:34:33.665175 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.665104 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 01 20:34:33.666133 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.666111 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:33.667999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.667903 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 01 20:34:33.667999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.667918 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rzs9n\"" Jun 01 20:34:33.667999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.667938 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 01 20:34:33.667999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.667906 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 01 20:34:33.670066 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.670047 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v8zqc"] Jun 01 20:34:33.673077 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.672712 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qf2m5"] Jun 01 20:34:33.739321 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.739244 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f014dc94-fbcf-4028-b18b-b7ecab010b40-tmp-dir\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.739321 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.739278 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv994\" (UniqueName: \"kubernetes.io/projected/24c4bd1b-12a5-4018-b753-83ec3ea7996c-kube-api-access-zv994\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:33.739321 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.739301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.739543 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.739387 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f014dc94-fbcf-4028-b18b-b7ecab010b40-config-volume\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.739543 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.739439 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cvlv\" (UniqueName: \"kubernetes.io/projected/f014dc94-fbcf-4028-b18b-b7ecab010b40-kube-api-access-9cvlv\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.739543 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.739512 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:33.827371 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.827336 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:34:33.827371 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.827365 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:33.830498 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.830356 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 01 20:34:33.830498 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.830371 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 01 20:34:33.830967 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.830857 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gkzmp\"" Jun 01 20:34:33.840460 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.840435 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:33.840572 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.840482 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f014dc94-fbcf-4028-b18b-b7ecab010b40-tmp-dir\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.840572 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.840510 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zv994\" (UniqueName: \"kubernetes.io/projected/24c4bd1b-12a5-4018-b753-83ec3ea7996c-kube-api-access-zv994\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:33.840572 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.840541 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 20:34:33.840727 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.840605 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert podName:24c4bd1b-12a5-4018-b753-83ec3ea7996c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:34.340585539 +0000 UTC m=+35.047958084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert") pod "ingress-canary-v8zqc" (UID: "24c4bd1b-12a5-4018-b753-83ec3ea7996c") : secret "canary-serving-cert" not found Jun 01 20:34:33.840727 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.840618 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 20:34:33.840727 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.840541 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.840727 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:33.840667 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls podName:f014dc94-fbcf-4028-b18b-b7ecab010b40 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:34.340650941 +0000 UTC m=+35.048023472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls") pod "dns-default-qf2m5" (UID: "f014dc94-fbcf-4028-b18b-b7ecab010b40") : secret "dns-default-metrics-tls" not found Jun 01 20:34:33.840727 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.840706 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f014dc94-fbcf-4028-b18b-b7ecab010b40-config-volume\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.840974 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.840730 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9cvlv\" (UniqueName: \"kubernetes.io/projected/f014dc94-fbcf-4028-b18b-b7ecab010b40-kube-api-access-9cvlv\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.840974 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.840863 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f014dc94-fbcf-4028-b18b-b7ecab010b40-tmp-dir\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.841176 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.841154 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f014dc94-fbcf-4028-b18b-b7ecab010b40-config-volume\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.850965 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.850938 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cvlv\" (UniqueName: \"kubernetes.io/projected/f014dc94-fbcf-4028-b18b-b7ecab010b40-kube-api-access-9cvlv\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:33.851087 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:33.851061 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv994\" (UniqueName: \"kubernetes.io/projected/24c4bd1b-12a5-4018-b753-83ec3ea7996c-kube-api-access-zv994\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:34.345991 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:34.345948 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:34.346282 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:34.345999 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:34.346282 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:34.346134 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 20:34:34.346282 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:34.346203 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert podName:24c4bd1b-12a5-4018-b753-83ec3ea7996c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:35.346183682 +0000 UTC m=+36.053556199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert") pod "ingress-canary-v8zqc" (UID: "24c4bd1b-12a5-4018-b753-83ec3ea7996c") : secret "canary-serving-cert" not found Jun 01 20:34:34.346282 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:34.346220 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 20:34:34.346489 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:34.346288 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls podName:f014dc94-fbcf-4028-b18b-b7ecab010b40 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:35.346269482 +0000 UTC m=+36.053642000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls") pod "dns-default-qf2m5" (UID: "f014dc94-fbcf-4028-b18b-b7ecab010b40") : secret "dns-default-metrics-tls" not found Jun 01 20:34:34.826755 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:34.826719 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:34:34.828851 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:34.828823 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 01 20:34:34.829202 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:34.828902 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 01 20:34:34.829414 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:34.829396 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tjj44\"" Jun 01 20:34:35.354216 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:35.354026 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:35.354393 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:35.354236 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:35.354393 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:35.354186 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 20:34:35.354393 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:35.354373 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert podName:24c4bd1b-12a5-4018-b753-83ec3ea7996c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:37.354355549 +0000 UTC m=+38.061728068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert") pod "ingress-canary-v8zqc" (UID: "24c4bd1b-12a5-4018-b753-83ec3ea7996c") : secret "canary-serving-cert" not found Jun 01 20:34:35.354393 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:35.354376 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 20:34:35.354637 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:35.354454 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls podName:f014dc94-fbcf-4028-b18b-b7ecab010b40 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:37.354437896 +0000 UTC m=+38.061810436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls") pod "dns-default-qf2m5" (UID: "f014dc94-fbcf-4028-b18b-b7ecab010b40") : secret "dns-default-metrics-tls" not found Jun 01 20:34:37.370159 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:37.370112 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:37.370607 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:37.370180 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:37.370607 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:37.370295 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 20:34:37.370607 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:37.370317 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 20:34:37.370607 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:37.370380 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert podName:24c4bd1b-12a5-4018-b753-83ec3ea7996c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:41.370358112 +0000 UTC m=+42.077730641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert") pod "ingress-canary-v8zqc" (UID: "24c4bd1b-12a5-4018-b753-83ec3ea7996c") : secret "canary-serving-cert" not found Jun 01 20:34:37.370607 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:37.370402 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls podName:f014dc94-fbcf-4028-b18b-b7ecab010b40 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:41.370390345 +0000 UTC m=+42.077762866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls") pod "dns-default-qf2m5" (UID: "f014dc94-fbcf-4028-b18b-b7ecab010b40") : secret "dns-default-metrics-tls" not found Jun 01 20:34:41.126486 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:41.126385 2559 generic.go:358] "Generic (PLEG): container finished" podID="9601c936-7a8f-401f-80b3-0e4a53bd248a" containerID="c5a8716bed375f9fce5eca9818d5ad86c6bd9b0d0d17dec8754df6dbc61c36b2" exitCode=0 Jun 01 20:34:41.126486 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:41.126445 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerDied","Data":"c5a8716bed375f9fce5eca9818d5ad86c6bd9b0d0d17dec8754df6dbc61c36b2"} Jun 01 20:34:41.403032 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:41.402992 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:41.403186 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:41.403044 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:41.403186 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:41.403130 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 20:34:41.403186 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:41.403134 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 20:34:41.403186 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:41.403186 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls podName:f014dc94-fbcf-4028-b18b-b7ecab010b40 nodeName:}" failed. No retries permitted until 2026-06-01 20:34:49.40316959 +0000 UTC m=+50.110542127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls") pod "dns-default-qf2m5" (UID: "f014dc94-fbcf-4028-b18b-b7ecab010b40") : secret "dns-default-metrics-tls" not found Jun 01 20:34:41.403314 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:41.403200 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert podName:24c4bd1b-12a5-4018-b753-83ec3ea7996c nodeName:}" failed. No retries permitted until 2026-06-01 20:34:49.403193927 +0000 UTC m=+50.110566443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert") pod "ingress-canary-v8zqc" (UID: "24c4bd1b-12a5-4018-b753-83ec3ea7996c") : secret "canary-serving-cert" not found Jun 01 20:34:42.131115 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:42.131068 2559 generic.go:358] "Generic (PLEG): container finished" podID="9601c936-7a8f-401f-80b3-0e4a53bd248a" containerID="964701b3175f09a139e946138a204e382dbcb07f4ca6f72620a122313f3da599" exitCode=0 Jun 01 20:34:42.131115 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:42.131118 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerDied","Data":"964701b3175f09a139e946138a204e382dbcb07f4ca6f72620a122313f3da599"} Jun 01 20:34:43.136171 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:43.136134 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pd28x" event={"ID":"9601c936-7a8f-401f-80b3-0e4a53bd248a","Type":"ContainerStarted","Data":"7eb14ed572f8ef4e05780b427fbe975761b89a43065f0b1a690ad0cc775b8e4d"} Jun 01 20:34:43.155302 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:43.155232 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pd28x" podStartSLOduration=6.094445243 podStartE2EDuration="44.155218898s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:34:02.366685297 +0000 UTC m=+3.074057817" lastFinishedPulling="2026-06-01 20:34:40.427458956 +0000 UTC m=+41.134831472" observedRunningTime="2026-06-01 20:34:43.153864012 +0000 UTC m=+43.861236549" watchObservedRunningTime="2026-06-01 20:34:43.155218898 +0000 UTC m=+43.862591435" Jun 01 20:34:44.324373 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:44.324333 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:44.327192 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:44.327169 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/752796d9-4fc8-4dd8-b91c-611bdc542833-original-pull-secret\") pod \"global-pull-secret-syncer-p8f5k\" (UID: \"752796d9-4fc8-4dd8-b91c-611bdc542833\") " pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:44.347278 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:44.347252 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p8f5k" Jun 01 20:34:44.504537 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:44.504497 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p8f5k"] Jun 01 20:34:44.508068 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:44.508037 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752796d9_4fc8_4dd8_b91c_611bdc542833.slice/crio-e1ce17754caa03d2f7e164f2fb890c9e4ca0391deb7f488510447bbbeb238a17 WatchSource:0}: Error finding container e1ce17754caa03d2f7e164f2fb890c9e4ca0391deb7f488510447bbbeb238a17: Status 404 returned error can't find the container with id e1ce17754caa03d2f7e164f2fb890c9e4ca0391deb7f488510447bbbeb238a17 Jun 01 20:34:45.140611 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:45.140573 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p8f5k" event={"ID":"752796d9-4fc8-4dd8-b91c-611bdc542833","Type":"ContainerStarted","Data":"e1ce17754caa03d2f7e164f2fb890c9e4ca0391deb7f488510447bbbeb238a17"} Jun 01 20:34:49.461226 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:49.461199 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:34:49.461545 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:49.461241 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:34:49.461545 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:49.461347 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 20:34:49.461545 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:49.461352 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 20:34:49.461545 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:49.461392 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls podName:f014dc94-fbcf-4028-b18b-b7ecab010b40 nodeName:}" failed. No retries permitted until 2026-06-01 20:35:05.461379163 +0000 UTC m=+66.168751678 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls") pod "dns-default-qf2m5" (UID: "f014dc94-fbcf-4028-b18b-b7ecab010b40") : secret "dns-default-metrics-tls" not found Jun 01 20:34:49.461545 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:34:49.461405 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert podName:24c4bd1b-12a5-4018-b753-83ec3ea7996c nodeName:}" failed. No retries permitted until 2026-06-01 20:35:05.461399307 +0000 UTC m=+66.168771822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert") pod "ingress-canary-v8zqc" (UID: "24c4bd1b-12a5-4018-b753-83ec3ea7996c") : secret "canary-serving-cert" not found Jun 01 20:34:50.151548 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:50.151514 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p8f5k" event={"ID":"752796d9-4fc8-4dd8-b91c-611bdc542833","Type":"ContainerStarted","Data":"0c9e8e65ff27ffca50318674df506eed5d7503d9ef8696a26e00554971be9aa8"} Jun 01 20:34:58.553023 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.552963 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p8f5k" podStartSLOduration=41.738510575 podStartE2EDuration="46.552945934s" podCreationTimestamp="2026-06-01 20:34:12 +0000 UTC" firstStartedPulling="2026-06-01 20:34:44.509702882 +0000 UTC m=+45.217075410" lastFinishedPulling="2026-06-01 20:34:49.324138248 +0000 UTC m=+50.031510769" observedRunningTime="2026-06-01 20:34:50.197569821 +0000 UTC m=+50.904942358" watchObservedRunningTime="2026-06-01 20:34:58.552945934 +0000 UTC m=+59.260318471" Jun 01 20:34:58.553408 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.553268 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz"] Jun 01 20:34:58.556576 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.556555 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" Jun 01 20:34:58.557070 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.557045 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp"] Jun 01 20:34:58.558562 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.558536 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-csl47\"" Jun 01 20:34:58.558679 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.558582 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 01 20:34:58.558679 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.558542 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 01 20:34:58.558973 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.558955 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 01 20:34:58.559014 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.558981 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 01 20:34:58.559691 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.559676 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.561280 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.561260 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 01 20:34:58.561530 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.561509 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 01 20:34:58.561530 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.561525 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 01 20:34:58.561704 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.561632 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 01 20:34:58.562741 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.562718 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz"] Jun 01 20:34:58.568567 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.568548 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp"] Jun 01 20:34:58.725943 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.725910 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dnwn\" (UniqueName: \"kubernetes.io/projected/22a4cbcb-a203-4a7b-afe0-aa6bde5c0288-kube-api-access-6dnwn\") pod \"managed-serviceaccount-addon-agent-6dddcff45d-xrpxz\" (UID: \"22a4cbcb-a203-4a7b-afe0-aa6bde5c0288\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" Jun 01 20:34:58.726116 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.725953 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-ca\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.726116 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.725970 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-hub\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.726116 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.725990 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.726116 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.726071 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.726116 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.726104 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.726277 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.726159 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6cgk\" (UniqueName: \"kubernetes.io/projected/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-kube-api-access-r6cgk\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.726277 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.726231 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22a4cbcb-a203-4a7b-afe0-aa6bde5c0288-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dddcff45d-xrpxz\" (UID: \"22a4cbcb-a203-4a7b-afe0-aa6bde5c0288\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" Jun 01 20:34:58.827246 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.827168 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-ca\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.827246 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.827205 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-hub\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.827246 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.827230 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.827501 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.827283 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.827501 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.827314 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.827501 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.827338 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r6cgk\" (UniqueName: \"kubernetes.io/projected/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-kube-api-access-r6cgk\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.827501 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.827489 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22a4cbcb-a203-4a7b-afe0-aa6bde5c0288-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dddcff45d-xrpxz\" (UID: \"22a4cbcb-a203-4a7b-afe0-aa6bde5c0288\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" Jun 01 20:34:58.827689 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.827584 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6dnwn\" (UniqueName: \"kubernetes.io/projected/22a4cbcb-a203-4a7b-afe0-aa6bde5c0288-kube-api-access-6dnwn\") pod \"managed-serviceaccount-addon-agent-6dddcff45d-xrpxz\" (UID: \"22a4cbcb-a203-4a7b-afe0-aa6bde5c0288\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" Jun 01 20:34:58.828148 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.828122 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.829843 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.829812 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-ca\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.829925 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.829867 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.830000 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.829980 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.830284 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.830268 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/22a4cbcb-a203-4a7b-afe0-aa6bde5c0288-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dddcff45d-xrpxz\" (UID: \"22a4cbcb-a203-4a7b-afe0-aa6bde5c0288\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" Jun 01 20:34:58.830320 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.830283 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-hub\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.836744 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.836720 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dnwn\" (UniqueName: \"kubernetes.io/projected/22a4cbcb-a203-4a7b-afe0-aa6bde5c0288-kube-api-access-6dnwn\") pod \"managed-serviceaccount-addon-agent-6dddcff45d-xrpxz\" (UID: \"22a4cbcb-a203-4a7b-afe0-aa6bde5c0288\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" Jun 01 20:34:58.836935 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.836920 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6cgk\" (UniqueName: \"kubernetes.io/projected/6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5-kube-api-access-r6cgk\") pod \"cluster-proxy-proxy-agent-59c9b48947-xqvjp\" (UID: \"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:58.875003 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.874955 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" Jun 01 20:34:58.880833 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:58.880807 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:34:59.018667 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:59.018630 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz"] Jun 01 20:34:59.024452 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:59.024404 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22a4cbcb_a203_4a7b_afe0_aa6bde5c0288.slice/crio-3f6029e74132826af950b7fd438333c07b4c02f3fa9d2bbce774f4024da9d89f WatchSource:0}: Error finding container 3f6029e74132826af950b7fd438333c07b4c02f3fa9d2bbce774f4024da9d89f: Status 404 returned error can't find the container with id 3f6029e74132826af950b7fd438333c07b4c02f3fa9d2bbce774f4024da9d89f Jun 01 20:34:59.036361 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:59.036338 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp"] Jun 01 20:34:59.039258 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:34:59.039237 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d1c2da2_6b95_424e_bdcc_d9f1281e0ed5.slice/crio-25fb4a71b0b39ad72621be03c0ad1943139c5a150bcb25c5ba11bc7a2abcdc82 WatchSource:0}: Error finding container 25fb4a71b0b39ad72621be03c0ad1943139c5a150bcb25c5ba11bc7a2abcdc82: Status 404 returned error can't find the container with id 25fb4a71b0b39ad72621be03c0ad1943139c5a150bcb25c5ba11bc7a2abcdc82 Jun 01 20:34:59.109708 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:59.109640 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-knvtx" Jun 01 20:34:59.170048 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:59.170010 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" event={"ID":"22a4cbcb-a203-4a7b-afe0-aa6bde5c0288","Type":"ContainerStarted","Data":"3f6029e74132826af950b7fd438333c07b4c02f3fa9d2bbce774f4024da9d89f"} Jun 01 20:34:59.170984 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:34:59.170959 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" event={"ID":"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5","Type":"ContainerStarted","Data":"25fb4a71b0b39ad72621be03c0ad1943139c5a150bcb25c5ba11bc7a2abcdc82"} Jun 01 20:35:04.182606 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:04.182564 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" event={"ID":"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5","Type":"ContainerStarted","Data":"269f64ade45fb71ac4fa153e6157bea2522ede12679dec218e7081afbd9213c3"} Jun 01 20:35:04.183772 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:04.183745 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" event={"ID":"22a4cbcb-a203-4a7b-afe0-aa6bde5c0288","Type":"ContainerStarted","Data":"2a0a15e3fca38626af4a7e6a6927e2f8112b81a15458c035063aa71fc1c25809"} Jun 01 20:35:04.196780 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:04.196739 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dddcff45d-xrpxz" podStartSLOduration=2.043348046 podStartE2EDuration="6.196726508s" podCreationTimestamp="2026-06-01 20:34:58 +0000 UTC" firstStartedPulling="2026-06-01 20:34:59.025949393 +0000 UTC m=+59.733321908" lastFinishedPulling="2026-06-01 20:35:03.17932785 +0000 UTC m=+63.886700370" observedRunningTime="2026-06-01 20:35:04.195757088 +0000 UTC m=+64.903129625" watchObservedRunningTime="2026-06-01 20:35:04.196726508 +0000 UTC m=+64.904099044" Jun 01 20:35:05.478438 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.478389 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:35:05.478947 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:05.478528 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 20:35:05.478947 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.478562 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:35:05.478947 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:05.478603 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert podName:24c4bd1b-12a5-4018-b753-83ec3ea7996c nodeName:}" failed. No retries permitted until 2026-06-01 20:35:37.478582716 +0000 UTC m=+98.185955255 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert") pod "ingress-canary-v8zqc" (UID: "24c4bd1b-12a5-4018-b753-83ec3ea7996c") : secret "canary-serving-cert" not found Jun 01 20:35:05.478947 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.478641 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:35:05.478947 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:05.478663 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 20:35:05.478947 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:05.478712 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls podName:f014dc94-fbcf-4028-b18b-b7ecab010b40 nodeName:}" failed. No retries permitted until 2026-06-01 20:35:37.478696437 +0000 UTC m=+98.186068960 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls") pod "dns-default-qf2m5" (UID: "f014dc94-fbcf-4028-b18b-b7ecab010b40") : secret "dns-default-metrics-tls" not found Jun 01 20:35:05.480519 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.480499 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 01 20:35:05.488959 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:05.488940 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 01 20:35:05.489071 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:05.488990 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs podName:818c75c1-e3d8-4a90-89de-ade8164c6d4a nodeName:}" failed. No retries permitted until 2026-06-01 20:36:09.48897421 +0000 UTC m=+130.196346738 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs") pod "network-metrics-daemon-z6xn6" (UID: "818c75c1-e3d8-4a90-89de-ade8164c6d4a") : secret "metrics-daemon-secret" not found Jun 01 20:35:05.579876 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.579844 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:35:05.581701 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.581674 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 01 20:35:05.593023 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.592994 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 01 20:35:05.604192 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.604171 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvblj\" (UniqueName: \"kubernetes.io/projected/20363d2d-83dd-498f-9629-231a4496db4c-kube-api-access-lvblj\") pod \"network-check-target-p6w46\" (UID: \"20363d2d-83dd-498f-9629-231a4496db4c\") " pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:35:05.740633 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.740553 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tjj44\"" Jun 01 20:35:05.748913 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.748892 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:35:05.949110 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:05.949074 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p6w46"] Jun 01 20:35:05.952892 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:35:05.952867 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20363d2d_83dd_498f_9629_231a4496db4c.slice/crio-17b0dff2665c87ea68f3c64fb46bbc9df07199812a9a7860fc6adde75ede76e2 WatchSource:0}: Error finding container 17b0dff2665c87ea68f3c64fb46bbc9df07199812a9a7860fc6adde75ede76e2: Status 404 returned error can't find the container with id 17b0dff2665c87ea68f3c64fb46bbc9df07199812a9a7860fc6adde75ede76e2 Jun 01 20:35:06.189575 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:06.189542 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p6w46" event={"ID":"20363d2d-83dd-498f-9629-231a4496db4c","Type":"ContainerStarted","Data":"17b0dff2665c87ea68f3c64fb46bbc9df07199812a9a7860fc6adde75ede76e2"} Jun 01 20:35:07.193787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:07.193748 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" event={"ID":"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5","Type":"ContainerStarted","Data":"a878b4d8514a549c6fe58ad8d2cc212265d86dc2ba000abebe0efba79165779d"} Jun 01 20:35:07.193787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:07.193792 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" event={"ID":"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5","Type":"ContainerStarted","Data":"683a35a5ff874d8fe8e2ae472d06fea6a2ccc782fabd0c8d5a69ec6a693fb6ef"} Jun 01 20:35:07.210835 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:07.210494 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" podStartSLOduration=1.509600944 podStartE2EDuration="9.210473733s" podCreationTimestamp="2026-06-01 20:34:58 +0000 UTC" firstStartedPulling="2026-06-01 20:34:59.040880235 +0000 UTC m=+59.748252751" lastFinishedPulling="2026-06-01 20:35:06.741753022 +0000 UTC m=+67.449125540" observedRunningTime="2026-06-01 20:35:07.208893496 +0000 UTC m=+67.916266034" watchObservedRunningTime="2026-06-01 20:35:07.210473733 +0000 UTC m=+67.917846272" Jun 01 20:35:10.203678 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:10.203644 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p6w46" event={"ID":"20363d2d-83dd-498f-9629-231a4496db4c","Type":"ContainerStarted","Data":"e40a0c8e4f5a6c906ed153b4fbe46820f9a43b5569961ea3a37e247190fa716b"} Jun 01 20:35:10.204043 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:10.203796 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:35:10.216751 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:10.216706 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p6w46" podStartSLOduration=67.751907129 podStartE2EDuration="1m11.21669108s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:35:05.955272204 +0000 UTC m=+66.662644724" lastFinishedPulling="2026-06-01 20:35:09.420056161 +0000 UTC m=+70.127428675" observedRunningTime="2026-06-01 20:35:10.216089734 +0000 UTC m=+70.923462271" watchObservedRunningTime="2026-06-01 20:35:10.21669108 +0000 UTC m=+70.924063620" Jun 01 20:35:37.498309 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:37.498214 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:35:37.498309 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:37.498264 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:35:37.498768 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:37.498362 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 20:35:37.498768 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:37.498368 2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 20:35:37.498768 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:37.498416 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls podName:f014dc94-fbcf-4028-b18b-b7ecab010b40 nodeName:}" failed. No retries permitted until 2026-06-01 20:36:41.498401025 +0000 UTC m=+162.205773539 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls") pod "dns-default-qf2m5" (UID: "f014dc94-fbcf-4028-b18b-b7ecab010b40") : secret "dns-default-metrics-tls" not found Jun 01 20:35:37.498768 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:35:37.498449 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert podName:24c4bd1b-12a5-4018-b753-83ec3ea7996c nodeName:}" failed. No retries permitted until 2026-06-01 20:36:41.498441477 +0000 UTC m=+162.205813992 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert") pod "ingress-canary-v8zqc" (UID: "24c4bd1b-12a5-4018-b753-83ec3ea7996c") : secret "canary-serving-cert" not found Jun 01 20:35:41.207731 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:35:41.207695 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p6w46" Jun 01 20:36:09.518891 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:09.518839 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:36:09.519387 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:36:09.518963 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 01 20:36:09.519387 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:36:09.519027 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs podName:818c75c1-e3d8-4a90-89de-ade8164c6d4a nodeName:}" failed. No retries permitted until 2026-06-01 20:38:11.519012377 +0000 UTC m=+252.226384892 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs") pod "network-metrics-daemon-z6xn6" (UID: "818c75c1-e3d8-4a90-89de-ade8164c6d4a") : secret "metrics-daemon-secret" not found Jun 01 20:36:19.005247 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:19.005218 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xp92g_d5979744-4da4-40e1-9001-477372c8d2c0/dns-node-resolver/0.log" Jun 01 20:36:19.805617 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:19.805590 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cx8lv_c0d1f181-3281-404c-a019-c7a99930d33d/node-ca/0.log" Jun 01 20:36:28.882134 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:28.882068 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" podUID="6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 01 20:36:36.676696 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:36:36.676649 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qf2m5" podUID="f014dc94-fbcf-4028-b18b-b7ecab010b40" Jun 01 20:36:36.682818 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:36:36.682790 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-v8zqc" podUID="24c4bd1b-12a5-4018-b753-83ec3ea7996c" Jun 01 20:36:36.841169 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:36:36.841125 2559 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-z6xn6" podUID="818c75c1-e3d8-4a90-89de-ade8164c6d4a" Jun 01 20:36:37.414571 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:37.414542 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qf2m5" Jun 01 20:36:37.414747 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:37.414545 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:36:38.882414 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:38.882369 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" podUID="6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 01 20:36:40.637570 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.637535 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t9ffl"] Jun 01 20:36:40.640775 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.640756 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.642626 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.642606 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 01 20:36:40.643047 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.643027 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 01 20:36:40.643047 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.643039 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 01 20:36:40.643148 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.643075 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hfgrl\"" Jun 01 20:36:40.643148 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.643119 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 01 20:36:40.651075 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.651053 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t9ffl"] Jun 01 20:36:40.740172 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.740135 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/93623e06-a28e-455a-aa34-3c7e446e8219-crio-socket\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.740350 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.740176 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93623e06-a28e-455a-aa34-3c7e446e8219-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.740350 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.740271 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87bnr\" (UniqueName: \"kubernetes.io/projected/93623e06-a28e-455a-aa34-3c7e446e8219-kube-api-access-87bnr\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.740350 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.740339 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/93623e06-a28e-455a-aa34-3c7e446e8219-data-volume\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.740491 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.740363 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93623e06-a28e-455a-aa34-3c7e446e8219-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.841512 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.841472 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/93623e06-a28e-455a-aa34-3c7e446e8219-crio-socket\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.841512 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.841517 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93623e06-a28e-455a-aa34-3c7e446e8219-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.841762 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.841569 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-87bnr\" (UniqueName: \"kubernetes.io/projected/93623e06-a28e-455a-aa34-3c7e446e8219-kube-api-access-87bnr\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.841762 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.841600 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/93623e06-a28e-455a-aa34-3c7e446e8219-crio-socket\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.841762 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.841633 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/93623e06-a28e-455a-aa34-3c7e446e8219-data-volume\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.841762 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.841662 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93623e06-a28e-455a-aa34-3c7e446e8219-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.842024 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.842002 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/93623e06-a28e-455a-aa34-3c7e446e8219-data-volume\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.842170 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.842154 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/93623e06-a28e-455a-aa34-3c7e446e8219-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.844126 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.844105 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/93623e06-a28e-455a-aa34-3c7e446e8219-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.857368 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.857339 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87bnr\" (UniqueName: \"kubernetes.io/projected/93623e06-a28e-455a-aa34-3c7e446e8219-kube-api-access-87bnr\") pod \"insights-runtime-extractor-t9ffl\" (UID: \"93623e06-a28e-455a-aa34-3c7e446e8219\") " pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:40.949951 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:40.949919 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t9ffl" Jun 01 20:36:41.065474 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.065381 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t9ffl"] Jun 01 20:36:41.069915 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:36:41.069887 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93623e06_a28e_455a_aa34_3c7e446e8219.slice/crio-c0275dcdf6060685c2a577574127a83228179cfc3d2ddbdffcf58e1cc7001df0 WatchSource:0}: Error finding container c0275dcdf6060685c2a577574127a83228179cfc3d2ddbdffcf58e1cc7001df0: Status 404 returned error can't find the container with id c0275dcdf6060685c2a577574127a83228179cfc3d2ddbdffcf58e1cc7001df0 Jun 01 20:36:41.424514 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.424479 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9ffl" event={"ID":"93623e06-a28e-455a-aa34-3c7e446e8219","Type":"ContainerStarted","Data":"3e3d6c6e4cb0dbf5f04aeb7f149692f144420f4c59f1e5de162fc7b321d7d736"} Jun 01 20:36:41.424670 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.424519 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9ffl" event={"ID":"93623e06-a28e-455a-aa34-3c7e446e8219","Type":"ContainerStarted","Data":"c0275dcdf6060685c2a577574127a83228179cfc3d2ddbdffcf58e1cc7001df0"} Jun 01 20:36:41.551528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.551492 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:36:41.551692 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.551542 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:36:41.553882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.553863 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f014dc94-fbcf-4028-b18b-b7ecab010b40-metrics-tls\") pod \"dns-default-qf2m5\" (UID: \"f014dc94-fbcf-4028-b18b-b7ecab010b40\") " pod="openshift-dns/dns-default-qf2m5" Jun 01 20:36:41.554003 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.553984 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24c4bd1b-12a5-4018-b753-83ec3ea7996c-cert\") pod \"ingress-canary-v8zqc\" (UID: \"24c4bd1b-12a5-4018-b753-83ec3ea7996c\") " pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:36:41.618721 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.618691 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6rq65\"" Jun 01 20:36:41.618721 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.618692 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rzs9n\"" Jun 01 20:36:41.625260 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.625237 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v8zqc" Jun 01 20:36:41.625366 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.625264 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qf2m5" Jun 01 20:36:41.758183 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.758149 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v8zqc"] Jun 01 20:36:41.761266 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:36:41.761240 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c4bd1b_12a5_4018_b753_83ec3ea7996c.slice/crio-6eac87b5e45a5405153266e50bbac05983ef70ad94fd708244e4f551c11e48a2 WatchSource:0}: Error finding container 6eac87b5e45a5405153266e50bbac05983ef70ad94fd708244e4f551c11e48a2: Status 404 returned error can't find the container with id 6eac87b5e45a5405153266e50bbac05983ef70ad94fd708244e4f551c11e48a2 Jun 01 20:36:41.774459 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:41.774432 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qf2m5"] Jun 01 20:36:41.777400 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:36:41.777376 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf014dc94_fbcf_4028_b18b_b7ecab010b40.slice/crio-fc6b47a4f3552ed3193e4305448c6bb988eb03c9335eff0fdc22c45230637609 WatchSource:0}: Error finding container fc6b47a4f3552ed3193e4305448c6bb988eb03c9335eff0fdc22c45230637609: Status 404 returned error can't find the container with id fc6b47a4f3552ed3193e4305448c6bb988eb03c9335eff0fdc22c45230637609 Jun 01 20:36:42.429223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:42.429164 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qf2m5" event={"ID":"f014dc94-fbcf-4028-b18b-b7ecab010b40","Type":"ContainerStarted","Data":"fc6b47a4f3552ed3193e4305448c6bb988eb03c9335eff0fdc22c45230637609"} Jun 01 20:36:42.431208 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:42.431161 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9ffl" event={"ID":"93623e06-a28e-455a-aa34-3c7e446e8219","Type":"ContainerStarted","Data":"2a0bb2af03dee4beda304f3ee82d3afa29f796c024d77e1348ef5d043c74db0a"} Jun 01 20:36:42.432997 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:42.432942 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v8zqc" event={"ID":"24c4bd1b-12a5-4018-b753-83ec3ea7996c","Type":"ContainerStarted","Data":"6eac87b5e45a5405153266e50bbac05983ef70ad94fd708244e4f551c11e48a2"} Jun 01 20:36:44.439760 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:44.439716 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v8zqc" event={"ID":"24c4bd1b-12a5-4018-b753-83ec3ea7996c","Type":"ContainerStarted","Data":"69c4d0d791bb2c603530de893026a263672d142f8340eaed18c7f85243ae5e32"} Jun 01 20:36:44.441498 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:44.441465 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qf2m5" event={"ID":"f014dc94-fbcf-4028-b18b-b7ecab010b40","Type":"ContainerStarted","Data":"80bfe944bc487f19576eba55077e45713c17359d0aaaff06091b63e984e8d1e9"} Jun 01 20:36:44.441609 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:44.441508 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qf2m5" event={"ID":"f014dc94-fbcf-4028-b18b-b7ecab010b40","Type":"ContainerStarted","Data":"0736d3bf6e378ea11b542715c8046c631986362dfd1c0062caf664710552ef8f"} Jun 01 20:36:44.441703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:44.441673 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qf2m5" Jun 01 20:36:44.443479 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:44.443456 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9ffl" event={"ID":"93623e06-a28e-455a-aa34-3c7e446e8219","Type":"ContainerStarted","Data":"cf3d488a141850e28d3de35b2a559133387197233d9d06fe0d449ac06aa0a36b"} Jun 01 20:36:44.452570 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:44.452519 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v8zqc" podStartSLOduration=128.956764358 podStartE2EDuration="2m11.452504479s" podCreationTimestamp="2026-06-01 20:34:33 +0000 UTC" firstStartedPulling="2026-06-01 20:36:41.763141 +0000 UTC m=+162.470513515" lastFinishedPulling="2026-06-01 20:36:44.258881104 +0000 UTC m=+164.966253636" observedRunningTime="2026-06-01 20:36:44.452103853 +0000 UTC m=+165.159476392" watchObservedRunningTime="2026-06-01 20:36:44.452504479 +0000 UTC m=+165.159877018" Jun 01 20:36:44.466921 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:44.466846 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t9ffl" podStartSLOduration=1.3746046889999999 podStartE2EDuration="4.466834144s" podCreationTimestamp="2026-06-01 20:36:40 +0000 UTC" firstStartedPulling="2026-06-01 20:36:41.121079614 +0000 UTC m=+161.828452130" lastFinishedPulling="2026-06-01 20:36:44.213309056 +0000 UTC m=+164.920681585" observedRunningTime="2026-06-01 20:36:44.465954262 +0000 UTC m=+165.173326800" watchObservedRunningTime="2026-06-01 20:36:44.466834144 +0000 UTC m=+165.174206681" Jun 01 20:36:44.479821 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:44.479779 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-qf2m5" podStartSLOduration=129.050805742 podStartE2EDuration="2m11.479766402s" podCreationTimestamp="2026-06-01 20:34:33 +0000 UTC" firstStartedPulling="2026-06-01 20:36:41.779050668 +0000 UTC m=+162.486423186" lastFinishedPulling="2026-06-01 20:36:44.208011318 +0000 UTC m=+164.915383846" observedRunningTime="2026-06-01 20:36:44.478867334 +0000 UTC m=+165.186239873" watchObservedRunningTime="2026-06-01 20:36:44.479766402 +0000 UTC m=+165.187138938" Jun 01 20:36:46.770624 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.770592 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-79cd4678d6-jxznk"] Jun 01 20:36:46.773454 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.773414 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.775718 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.775693 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hg7dv\"" Jun 01 20:36:46.775827 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.775808 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 01 20:36:46.775893 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.775820 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 01 20:36:46.775893 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.775832 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 01 20:36:46.775893 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.775850 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 01 20:36:46.776191 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.776172 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 01 20:36:46.776343 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.776176 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 01 20:36:46.776343 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.776209 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 01 20:36:46.781898 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.781879 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79cd4678d6-jxznk"] Jun 01 20:36:46.897130 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.897096 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-oauth-config\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.897130 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.897132 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-service-ca\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.897325 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.897152 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-oauth-serving-cert\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.897325 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.897251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdlzw\" (UniqueName: \"kubernetes.io/projected/666b226e-5777-43a7-b8a1-0bfbba372ed8-kube-api-access-xdlzw\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.897325 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.897290 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-config\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.897416 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.897362 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-serving-cert\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.998326 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.998299 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-oauth-config\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.998326 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.998328 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-service-ca\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.998479 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.998345 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-oauth-serving-cert\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.998479 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.998399 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xdlzw\" (UniqueName: \"kubernetes.io/projected/666b226e-5777-43a7-b8a1-0bfbba372ed8-kube-api-access-xdlzw\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.998479 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.998467 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-config\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.998632 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.998517 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-serving-cert\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.999143 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.999120 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-service-ca\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.999253 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.999192 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-config\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:46.999253 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:46.999212 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-oauth-serving-cert\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:47.000888 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.000859 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-oauth-config\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:47.000977 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.000920 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-serving-cert\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:47.005011 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.004990 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdlzw\" (UniqueName: \"kubernetes.io/projected/666b226e-5777-43a7-b8a1-0bfbba372ed8-kube-api-access-xdlzw\") pod \"console-79cd4678d6-jxznk\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:47.082566 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.082496 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:47.199726 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.199696 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79cd4678d6-jxznk"] Jun 01 20:36:47.203059 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:36:47.203015 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod666b226e_5777_43a7_b8a1_0bfbba372ed8.slice/crio-74b0cc17ee5bc13b6447597a58bab838d6ca80e0f23e99130dd16fa7ab5944b4 WatchSource:0}: Error finding container 74b0cc17ee5bc13b6447597a58bab838d6ca80e0f23e99130dd16fa7ab5944b4: Status 404 returned error can't find the container with id 74b0cc17ee5bc13b6447597a58bab838d6ca80e0f23e99130dd16fa7ab5944b4 Jun 01 20:36:47.451443 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.451401 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cd4678d6-jxznk" event={"ID":"666b226e-5777-43a7-b8a1-0bfbba372ed8","Type":"ContainerStarted","Data":"74b0cc17ee5bc13b6447597a58bab838d6ca80e0f23e99130dd16fa7ab5944b4"} Jun 01 20:36:47.952514 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.952476 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp"] Jun 01 20:36:47.955366 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.955346 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" Jun 01 20:36:47.957199 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.957174 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 01 20:36:47.957316 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.957305 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-g8f6c\"" Jun 01 20:36:47.966192 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:47.966171 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp"] Jun 01 20:36:48.109085 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.109045 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/188b747c-a516-4422-af9e-b878c6b2bbf2-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-d6cpp\" (UID: \"188b747c-a516-4422-af9e-b878c6b2bbf2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" Jun 01 20:36:48.209662 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.209573 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/188b747c-a516-4422-af9e-b878c6b2bbf2-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-d6cpp\" (UID: \"188b747c-a516-4422-af9e-b878c6b2bbf2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" Jun 01 20:36:48.209819 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:36:48.209748 2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 01 20:36:48.209883 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:36:48.209832 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/188b747c-a516-4422-af9e-b878c6b2bbf2-tls-certificates podName:188b747c-a516-4422-af9e-b878c6b2bbf2 nodeName:}" failed. No retries permitted until 2026-06-01 20:36:48.709808424 +0000 UTC m=+169.417180944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/188b747c-a516-4422-af9e-b878c6b2bbf2-tls-certificates") pod "prometheus-operator-admission-webhook-59b7f8756b-d6cpp" (UID: "188b747c-a516-4422-af9e-b878c6b2bbf2") : secret "prometheus-operator-admission-webhook-tls" not found Jun 01 20:36:48.714477 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.714440 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/188b747c-a516-4422-af9e-b878c6b2bbf2-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-d6cpp\" (UID: \"188b747c-a516-4422-af9e-b878c6b2bbf2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" Jun 01 20:36:48.717174 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.717138 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/188b747c-a516-4422-af9e-b878c6b2bbf2-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-d6cpp\" (UID: \"188b747c-a516-4422-af9e-b878c6b2bbf2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" Jun 01 20:36:48.865824 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.865788 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" Jun 01 20:36:48.882832 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.882798 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" podUID="6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 01 20:36:48.882945 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.882881 2559 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" Jun 01 20:36:48.883395 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.883362 2559 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"a878b4d8514a549c6fe58ad8d2cc212265d86dc2ba000abebe0efba79165779d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 01 20:36:48.883469 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.883445 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" podUID="6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5" containerName="service-proxy" containerID="cri-o://a878b4d8514a549c6fe58ad8d2cc212265d86dc2ba000abebe0efba79165779d" gracePeriod=30 Jun 01 20:36:48.996857 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:48.996770 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp"] Jun 01 20:36:49.000721 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:36:49.000692 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188b747c_a516_4422_af9e_b878c6b2bbf2.slice/crio-f7d1b3c12a43c63374fe0ac57a4bbea7109623e6929b2b7d5fba00ae8f3fa42d WatchSource:0}: Error finding container f7d1b3c12a43c63374fe0ac57a4bbea7109623e6929b2b7d5fba00ae8f3fa42d: Status 404 returned error can't find the container with id f7d1b3c12a43c63374fe0ac57a4bbea7109623e6929b2b7d5fba00ae8f3fa42d Jun 01 20:36:49.459553 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:49.459509 2559 generic.go:358] "Generic (PLEG): container finished" podID="6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5" containerID="a878b4d8514a549c6fe58ad8d2cc212265d86dc2ba000abebe0efba79165779d" exitCode=2 Jun 01 20:36:49.459759 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:49.459650 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" event={"ID":"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5","Type":"ContainerDied","Data":"a878b4d8514a549c6fe58ad8d2cc212265d86dc2ba000abebe0efba79165779d"} Jun 01 20:36:49.459759 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:49.459682 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59c9b48947-xqvjp" event={"ID":"6d1c2da2-6b95-424e-bdcc-d9f1281e0ed5","Type":"ContainerStarted","Data":"f8b3a130ab23f0117476afa49de83a3a8f45e0b733b9c502ca765b6dbfb67c76"} Jun 01 20:36:49.461152 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:49.461118 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" event={"ID":"188b747c-a516-4422-af9e-b878c6b2bbf2","Type":"ContainerStarted","Data":"f7d1b3c12a43c63374fe0ac57a4bbea7109623e6929b2b7d5fba00ae8f3fa42d"} Jun 01 20:36:51.467275 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:51.467240 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" event={"ID":"188b747c-a516-4422-af9e-b878c6b2bbf2","Type":"ContainerStarted","Data":"ba84d64fa48895893364f74067c2b4a3c32d60accd073d34f140cf1aed54027d"} Jun 01 20:36:51.467786 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:51.467498 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" Jun 01 20:36:51.468603 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:51.468578 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cd4678d6-jxznk" event={"ID":"666b226e-5777-43a7-b8a1-0bfbba372ed8","Type":"ContainerStarted","Data":"ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa"} Jun 01 20:36:51.472368 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:51.472351 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" Jun 01 20:36:51.479651 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:51.479612 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-d6cpp" podStartSLOduration=2.683584955 podStartE2EDuration="4.479600711s" podCreationTimestamp="2026-06-01 20:36:47 +0000 UTC" firstStartedPulling="2026-06-01 20:36:49.002946838 +0000 UTC m=+169.710319360" lastFinishedPulling="2026-06-01 20:36:50.798962598 +0000 UTC m=+171.506335116" observedRunningTime="2026-06-01 20:36:51.479181466 +0000 UTC m=+172.186554003" watchObservedRunningTime="2026-06-01 20:36:51.479600711 +0000 UTC m=+172.186973265" Jun 01 20:36:51.492946 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:51.492911 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-79cd4678d6-jxznk" podStartSLOduration=1.897621469 podStartE2EDuration="5.492900586s" podCreationTimestamp="2026-06-01 20:36:46 +0000 UTC" firstStartedPulling="2026-06-01 20:36:47.205515775 +0000 UTC m=+167.912888305" lastFinishedPulling="2026-06-01 20:36:50.800794892 +0000 UTC m=+171.508167422" observedRunningTime="2026-06-01 20:36:51.491731933 +0000 UTC m=+172.199104470" watchObservedRunningTime="2026-06-01 20:36:51.492900586 +0000 UTC m=+172.200273122" Jun 01 20:36:51.826745 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:51.826662 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:36:54.448226 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:54.448191 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qf2m5" Jun 01 20:36:56.364652 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.364620 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lxgxd"] Jun 01 20:36:56.366808 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.366789 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.368748 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.368723 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 01 20:36:56.368859 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.368776 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 01 20:36:56.368859 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.368791 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-scssc\"" Jun 01 20:36:56.368859 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.368835 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 01 20:36:56.368859 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.368847 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 01 20:36:56.369216 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.369200 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 01 20:36:56.369299 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.369257 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 01 20:36:56.472192 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.472157 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ddae794-85d0-4260-be7c-7f486ab7a23f-root\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.472192 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.472191 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ddae794-85d0-4260-be7c-7f486ab7a23f-sys\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.472371 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.472217 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-tls\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.472371 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.472263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.472371 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.472337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.472371 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.472362 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-textfile\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.472560 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.472385 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-wtmp\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.472560 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.472404 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjrkz\" (UniqueName: \"kubernetes.io/projected/5ddae794-85d0-4260-be7c-7f486ab7a23f-kube-api-access-gjrkz\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.472560 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.472475 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ddae794-85d0-4260-be7c-7f486ab7a23f-metrics-client-ca\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573178 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573142 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573342 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573190 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-textfile\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573342 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573223 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-wtmp\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573342 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573240 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gjrkz\" (UniqueName: \"kubernetes.io/projected/5ddae794-85d0-4260-be7c-7f486ab7a23f-kube-api-access-gjrkz\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573342 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573261 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ddae794-85d0-4260-be7c-7f486ab7a23f-metrics-client-ca\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573342 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573303 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ddae794-85d0-4260-be7c-7f486ab7a23f-root\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573342 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573328 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ddae794-85d0-4260-be7c-7f486ab7a23f-sys\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573644 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573357 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-tls\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573644 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573386 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573644 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573398 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ddae794-85d0-4260-be7c-7f486ab7a23f-sys\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573644 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573400 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-wtmp\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573644 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573400 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ddae794-85d0-4260-be7c-7f486ab7a23f-root\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573803 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573678 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-textfile\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573844 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573802 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.573950 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.573930 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ddae794-85d0-4260-be7c-7f486ab7a23f-metrics-client-ca\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.575698 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.575680 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-tls\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.575794 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.575757 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ddae794-85d0-4260-be7c-7f486ab7a23f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.579393 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.579372 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjrkz\" (UniqueName: \"kubernetes.io/projected/5ddae794-85d0-4260-be7c-7f486ab7a23f-kube-api-access-gjrkz\") pod \"node-exporter-lxgxd\" (UID: \"5ddae794-85d0-4260-be7c-7f486ab7a23f\") " pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.675926 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:56.675893 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxgxd" Jun 01 20:36:56.683485 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:36:56.683457 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ddae794_85d0_4260_be7c_7f486ab7a23f.slice/crio-c933532324f8133e347b35adbad2312d6871e3519045ba522c96054fb2d122a1 WatchSource:0}: Error finding container c933532324f8133e347b35adbad2312d6871e3519045ba522c96054fb2d122a1: Status 404 returned error can't find the container with id c933532324f8133e347b35adbad2312d6871e3519045ba522c96054fb2d122a1 Jun 01 20:36:57.083292 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.083216 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:57.083292 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.083255 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:57.088288 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.088260 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:57.425395 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.424866 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 01 20:36:57.428044 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.428016 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.429966 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.429897 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 01 20:36:57.429966 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.429921 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 01 20:36:57.430190 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.429973 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c27qh\"" Jun 01 20:36:57.430190 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.429898 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 01 20:36:57.430521 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.430493 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 01 20:36:57.430646 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.430557 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 01 20:36:57.430646 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.430566 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 01 20:36:57.430646 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.430586 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 01 20:36:57.430941 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.430924 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 01 20:36:57.430941 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.430939 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 01 20:36:57.482294 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.482257 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 01 20:36:57.485444 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.485395 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxgxd" event={"ID":"5ddae794-85d0-4260-be7c-7f486ab7a23f","Type":"ContainerStarted","Data":"c933532324f8133e347b35adbad2312d6871e3519045ba522c96054fb2d122a1"} Jun 01 20:36:57.490975 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.490950 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:36:57.582581 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.582560 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/427be6f2-acd6-4964-b6b4-4da497d87312-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.582667 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.582595 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/427be6f2-acd6-4964-b6b4-4da497d87312-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.582667 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.582623 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.582772 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.582707 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/427be6f2-acd6-4964-b6b4-4da497d87312-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.582846 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.582823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-config-volume\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.582901 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.582856 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/427be6f2-acd6-4964-b6b4-4da497d87312-tls-assets\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.582954 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.582911 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-web-config\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.582954 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.582947 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.583143 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.583119 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vwjp\" (UniqueName: \"kubernetes.io/projected/427be6f2-acd6-4964-b6b4-4da497d87312-kube-api-access-2vwjp\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.583246 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.583163 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/427be6f2-acd6-4964-b6b4-4da497d87312-config-out\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.583246 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.583203 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.583246 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.583234 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.583399 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.583257 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.683923 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.683835 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-config-volume\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.683923 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.683873 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/427be6f2-acd6-4964-b6b4-4da497d87312-tls-assets\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.683923 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.683894 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-web-config\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.683923 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.683910 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.684287 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.683937 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2vwjp\" (UniqueName: \"kubernetes.io/projected/427be6f2-acd6-4964-b6b4-4da497d87312-kube-api-access-2vwjp\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.684287 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.684081 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/427be6f2-acd6-4964-b6b4-4da497d87312-config-out\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.684287 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.684127 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.684287 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.684160 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.684287 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.684191 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.684287 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.684265 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/427be6f2-acd6-4964-b6b4-4da497d87312-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.684640 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.684293 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/427be6f2-acd6-4964-b6b4-4da497d87312-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.684640 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.684323 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.684640 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.684362 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/427be6f2-acd6-4964-b6b4-4da497d87312-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.686882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.685336 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/427be6f2-acd6-4964-b6b4-4da497d87312-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.686882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.685377 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/427be6f2-acd6-4964-b6b4-4da497d87312-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.686882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.685965 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/427be6f2-acd6-4964-b6b4-4da497d87312-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.686882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.686754 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/427be6f2-acd6-4964-b6b4-4da497d87312-config-out\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.688153 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.688123 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/427be6f2-acd6-4964-b6b4-4da497d87312-tls-assets\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.688277 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.688209 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-web-config\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.688277 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.688267 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.688395 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.688333 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.688395 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.688372 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-config-volume\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.688851 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.688827 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.688851 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.688846 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.689004 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.688946 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/427be6f2-acd6-4964-b6b4-4da497d87312-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.691386 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.691360 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vwjp\" (UniqueName: \"kubernetes.io/projected/427be6f2-acd6-4964-b6b4-4da497d87312-kube-api-access-2vwjp\") pod \"alertmanager-main-0\" (UID: \"427be6f2-acd6-4964-b6b4-4da497d87312\") " pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.739119 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.739091 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 01 20:36:57.865338 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:57.865305 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 01 20:36:57.868759 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:36:57.868730 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427be6f2_acd6_4964_b6b4_4da497d87312.slice/crio-dd7bbbf5e04a003cae29d97625d2c7e23b58fc2fd948fd5f9a42dcfcfad8a00f WatchSource:0}: Error finding container dd7bbbf5e04a003cae29d97625d2c7e23b58fc2fd948fd5f9a42dcfcfad8a00f: Status 404 returned error can't find the container with id dd7bbbf5e04a003cae29d97625d2c7e23b58fc2fd948fd5f9a42dcfcfad8a00f Jun 01 20:36:58.488914 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:58.488875 2559 generic.go:358] "Generic (PLEG): container finished" podID="5ddae794-85d0-4260-be7c-7f486ab7a23f" containerID="37a45104902dae91b9b87495e0828fdf36962f11b768b5181afaeafa38b309dd" exitCode=0 Jun 01 20:36:58.489369 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:58.488974 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxgxd" event={"ID":"5ddae794-85d0-4260-be7c-7f486ab7a23f","Type":"ContainerDied","Data":"37a45104902dae91b9b87495e0828fdf36962f11b768b5181afaeafa38b309dd"} Jun 01 20:36:58.490661 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:58.490627 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"427be6f2-acd6-4964-b6b4-4da497d87312","Type":"ContainerStarted","Data":"dd7bbbf5e04a003cae29d97625d2c7e23b58fc2fd948fd5f9a42dcfcfad8a00f"} Jun 01 20:36:59.494815 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:59.494779 2559 generic.go:358] "Generic (PLEG): container finished" podID="427be6f2-acd6-4964-b6b4-4da497d87312" containerID="db44e2a91fc77b7542d81fabf0428de3444f4cef2e1b67110d7b6a2130dc4a07" exitCode=0 Jun 01 20:36:59.495237 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:59.494866 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"427be6f2-acd6-4964-b6b4-4da497d87312","Type":"ContainerDied","Data":"db44e2a91fc77b7542d81fabf0428de3444f4cef2e1b67110d7b6a2130dc4a07"} Jun 01 20:36:59.496943 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:59.496920 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxgxd" event={"ID":"5ddae794-85d0-4260-be7c-7f486ab7a23f","Type":"ContainerStarted","Data":"2cf05294982aed8904b7f1aa4546ff224a41e6a0cabad35f518abe059826a61d"} Jun 01 20:36:59.497031 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:59.496950 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxgxd" event={"ID":"5ddae794-85d0-4260-be7c-7f486ab7a23f","Type":"ContainerStarted","Data":"fcda61909d31cbdcf6615ee14f8ed0a52145f1d1ccff29643e6f785a68f0b89a"} Jun 01 20:36:59.531882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:36:59.531838 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lxgxd" podStartSLOduration=2.64763928 podStartE2EDuration="3.531825683s" podCreationTimestamp="2026-06-01 20:36:56 +0000 UTC" firstStartedPulling="2026-06-01 20:36:56.685445939 +0000 UTC m=+177.392818453" lastFinishedPulling="2026-06-01 20:36:57.569632323 +0000 UTC m=+178.277004856" observedRunningTime="2026-06-01 20:36:59.531575676 +0000 UTC m=+180.238948214" watchObservedRunningTime="2026-06-01 20:36:59.531825683 +0000 UTC m=+180.239198220" Jun 01 20:37:00.667256 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.667219 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z"] Jun 01 20:37:00.670312 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.670292 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.672333 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.672310 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-wf6r4\"" Jun 01 20:37:00.672813 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.672792 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 01 20:37:00.672813 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.672803 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 01 20:37:00.673016 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.672820 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 01 20:37:00.673016 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.672804 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-agi8epkbqteq5\"" Jun 01 20:37:00.673016 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.672803 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 01 20:37:00.679270 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.679246 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z"] Jun 01 20:37:00.710105 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.710073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1654d8b6-07a6-4231-92c2-3b16961540b0-client-ca-bundle\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.710244 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.710121 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1654d8b6-07a6-4231-92c2-3b16961540b0-metrics-server-audit-profiles\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.710244 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.710150 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m5lp\" (UniqueName: \"kubernetes.io/projected/1654d8b6-07a6-4231-92c2-3b16961540b0-kube-api-access-2m5lp\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.710335 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.710249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1654d8b6-07a6-4231-92c2-3b16961540b0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.710335 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.710301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1654d8b6-07a6-4231-92c2-3b16961540b0-secret-metrics-server-client-certs\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.710455 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.710359 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1654d8b6-07a6-4231-92c2-3b16961540b0-secret-metrics-server-tls\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.710455 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.710391 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1654d8b6-07a6-4231-92c2-3b16961540b0-audit-log\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.810910 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.810871 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1654d8b6-07a6-4231-92c2-3b16961540b0-secret-metrics-server-tls\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.810910 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.810912 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1654d8b6-07a6-4231-92c2-3b16961540b0-audit-log\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.811151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.810939 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1654d8b6-07a6-4231-92c2-3b16961540b0-client-ca-bundle\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.811151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.810963 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1654d8b6-07a6-4231-92c2-3b16961540b0-metrics-server-audit-profiles\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.811151 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.811078 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2m5lp\" (UniqueName: \"kubernetes.io/projected/1654d8b6-07a6-4231-92c2-3b16961540b0-kube-api-access-2m5lp\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.811312 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.811159 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1654d8b6-07a6-4231-92c2-3b16961540b0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.811312 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.811200 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1654d8b6-07a6-4231-92c2-3b16961540b0-secret-metrics-server-client-certs\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.811471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.811345 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1654d8b6-07a6-4231-92c2-3b16961540b0-audit-log\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.811944 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.811923 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1654d8b6-07a6-4231-92c2-3b16961540b0-metrics-server-audit-profiles\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.812048 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.811922 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1654d8b6-07a6-4231-92c2-3b16961540b0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.813406 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.813385 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1654d8b6-07a6-4231-92c2-3b16961540b0-secret-metrics-server-tls\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.813520 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.813503 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1654d8b6-07a6-4231-92c2-3b16961540b0-secret-metrics-server-client-certs\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.813565 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.813540 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1654d8b6-07a6-4231-92c2-3b16961540b0-client-ca-bundle\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.818970 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.818941 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m5lp\" (UniqueName: \"kubernetes.io/projected/1654d8b6-07a6-4231-92c2-3b16961540b0-kube-api-access-2m5lp\") pod \"metrics-server-5fcb55cc7d-2xs4z\" (UID: \"1654d8b6-07a6-4231-92c2-3b16961540b0\") " pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:00.980758 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:00.980684 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:01.107200 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:01.107170 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z"] Jun 01 20:37:01.112301 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:37:01.112274 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1654d8b6_07a6_4231_92c2_3b16961540b0.slice/crio-506b3cd8dcd8c16ba4ddef45288d200655353a749955cab2d3191834c3c170bd WatchSource:0}: Error finding container 506b3cd8dcd8c16ba4ddef45288d200655353a749955cab2d3191834c3c170bd: Status 404 returned error can't find the container with id 506b3cd8dcd8c16ba4ddef45288d200655353a749955cab2d3191834c3c170bd Jun 01 20:37:01.503437 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:01.503389 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" event={"ID":"1654d8b6-07a6-4231-92c2-3b16961540b0","Type":"ContainerStarted","Data":"506b3cd8dcd8c16ba4ddef45288d200655353a749955cab2d3191834c3c170bd"} Jun 01 20:37:02.508641 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:02.508602 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"427be6f2-acd6-4964-b6b4-4da497d87312","Type":"ContainerStarted","Data":"d0f3275a3db46d5e885d0b5452193988d825695781c3f019bd8d56145a67bde1"} Jun 01 20:37:02.508641 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:02.508638 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"427be6f2-acd6-4964-b6b4-4da497d87312","Type":"ContainerStarted","Data":"a6de4d323be7976ccf98ab51bd108a40e74bd1250c65e2dc533258039bd08902"} Jun 01 20:37:02.509032 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:02.508653 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"427be6f2-acd6-4964-b6b4-4da497d87312","Type":"ContainerStarted","Data":"9b5b1bbc60021c31c26fa6b233edd9f0e0aed5449a59d7422a853fb0e2e9e14f"} Jun 01 20:37:02.509032 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:02.508665 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"427be6f2-acd6-4964-b6b4-4da497d87312","Type":"ContainerStarted","Data":"28e626a926625bb4a6e8fd765ab48bc3e0f2d5654b00b20587b7c8ee78724ec9"} Jun 01 20:37:02.509032 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:02.508676 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"427be6f2-acd6-4964-b6b4-4da497d87312","Type":"ContainerStarted","Data":"1a294abc6caeb1d481d4c2bbbb3bfc05b75f08607ca5b0e337e35a41f7db7f70"} Jun 01 20:37:03.512285 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:03.512241 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" event={"ID":"1654d8b6-07a6-4231-92c2-3b16961540b0","Type":"ContainerStarted","Data":"bf085b2c29d89e6853f3cc2fdd2c31c54dab858c0ca96ee0bb48e9b11e6bc03b"} Jun 01 20:37:03.514976 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:03.514950 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"427be6f2-acd6-4964-b6b4-4da497d87312","Type":"ContainerStarted","Data":"8ee6b8dc46f7604562cb2fbdd29ec4220f39ce3ba9a7156f93a2cd91712ae6b0"} Jun 01 20:37:03.528534 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:03.528483 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" podStartSLOduration=1.707870303 podStartE2EDuration="3.528467674s" podCreationTimestamp="2026-06-01 20:37:00 +0000 UTC" firstStartedPulling="2026-06-01 20:37:01.114672874 +0000 UTC m=+181.822045404" lastFinishedPulling="2026-06-01 20:37:02.935270252 +0000 UTC m=+183.642642775" observedRunningTime="2026-06-01 20:37:03.5271972 +0000 UTC m=+184.234569737" watchObservedRunningTime="2026-06-01 20:37:03.528467674 +0000 UTC m=+184.235840214" Jun 01 20:37:03.552488 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:03.552439 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.490266261 podStartE2EDuration="6.55240498s" podCreationTimestamp="2026-06-01 20:36:57 +0000 UTC" firstStartedPulling="2026-06-01 20:36:57.870534353 +0000 UTC m=+178.577906871" lastFinishedPulling="2026-06-01 20:37:02.932673075 +0000 UTC m=+183.640045590" observedRunningTime="2026-06-01 20:37:03.550246494 +0000 UTC m=+184.257619031" watchObservedRunningTime="2026-06-01 20:37:03.55240498 +0000 UTC m=+184.259777520" Jun 01 20:37:12.965139 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:12.965105 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79cd4678d6-jxznk"] Jun 01 20:37:20.981758 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:20.981725 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:20.982092 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:20.981794 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:32.039726 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:32.039685 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v8zqc_24c4bd1b-12a5-4018-b753-83ec3ea7996c/serve-healthcheck-canary/0.log" Jun 01 20:37:37.984280 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:37.984220 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-79cd4678d6-jxznk" podUID="666b226e-5777-43a7-b8a1-0bfbba372ed8" containerName="console" containerID="cri-o://ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa" gracePeriod=15 Jun 01 20:37:38.232214 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.232193 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79cd4678d6-jxznk_666b226e-5777-43a7-b8a1-0bfbba372ed8/console/0.log" Jun 01 20:37:38.232330 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.232263 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:37:38.326584 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.326495 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-oauth-serving-cert\") pod \"666b226e-5777-43a7-b8a1-0bfbba372ed8\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " Jun 01 20:37:38.326584 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.326535 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdlzw\" (UniqueName: \"kubernetes.io/projected/666b226e-5777-43a7-b8a1-0bfbba372ed8-kube-api-access-xdlzw\") pod \"666b226e-5777-43a7-b8a1-0bfbba372ed8\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " Jun 01 20:37:38.326584 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.326565 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-service-ca\") pod \"666b226e-5777-43a7-b8a1-0bfbba372ed8\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " Jun 01 20:37:38.326584 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.326586 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-config\") pod \"666b226e-5777-43a7-b8a1-0bfbba372ed8\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " Jun 01 20:37:38.327355 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.326969 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-oauth-config\") pod \"666b226e-5777-43a7-b8a1-0bfbba372ed8\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " Jun 01 20:37:38.327355 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.327202 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "666b226e-5777-43a7-b8a1-0bfbba372ed8" (UID: "666b226e-5777-43a7-b8a1-0bfbba372ed8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 01 20:37:38.327355 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.327229 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-serving-cert\") pod \"666b226e-5777-43a7-b8a1-0bfbba372ed8\" (UID: \"666b226e-5777-43a7-b8a1-0bfbba372ed8\") " Jun 01 20:37:38.327355 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.327287 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-config" (OuterVolumeSpecName: "console-config") pod "666b226e-5777-43a7-b8a1-0bfbba372ed8" (UID: "666b226e-5777-43a7-b8a1-0bfbba372ed8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 01 20:37:38.327355 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.327298 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-service-ca" (OuterVolumeSpecName: "service-ca") pod "666b226e-5777-43a7-b8a1-0bfbba372ed8" (UID: "666b226e-5777-43a7-b8a1-0bfbba372ed8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 01 20:37:38.330320 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.328562 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-oauth-serving-cert\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:37:38.330320 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.328590 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-service-ca\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:37:38.330320 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.328607 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-config\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:37:38.331655 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.331631 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/666b226e-5777-43a7-b8a1-0bfbba372ed8-kube-api-access-xdlzw" (OuterVolumeSpecName: "kube-api-access-xdlzw") pod "666b226e-5777-43a7-b8a1-0bfbba372ed8" (UID: "666b226e-5777-43a7-b8a1-0bfbba372ed8"). InnerVolumeSpecName "kube-api-access-xdlzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 20:37:38.333067 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.333026 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "666b226e-5777-43a7-b8a1-0bfbba372ed8" (UID: "666b226e-5777-43a7-b8a1-0bfbba372ed8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 01 20:37:38.333278 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.333256 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "666b226e-5777-43a7-b8a1-0bfbba372ed8" (UID: "666b226e-5777-43a7-b8a1-0bfbba372ed8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 01 20:37:38.429847 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.429798 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdlzw\" (UniqueName: \"kubernetes.io/projected/666b226e-5777-43a7-b8a1-0bfbba372ed8-kube-api-access-xdlzw\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:37:38.429847 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.429845 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-oauth-config\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:37:38.429847 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.429855 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/666b226e-5777-43a7-b8a1-0bfbba372ed8-console-serving-cert\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:37:38.605287 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.605214 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79cd4678d6-jxznk_666b226e-5777-43a7-b8a1-0bfbba372ed8/console/0.log" Jun 01 20:37:38.605287 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.605255 2559 generic.go:358] "Generic (PLEG): container finished" podID="666b226e-5777-43a7-b8a1-0bfbba372ed8" containerID="ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa" exitCode=2 Jun 01 20:37:38.605486 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.605353 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cd4678d6-jxznk" Jun 01 20:37:38.605486 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.605349 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cd4678d6-jxznk" event={"ID":"666b226e-5777-43a7-b8a1-0bfbba372ed8","Type":"ContainerDied","Data":"ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa"} Jun 01 20:37:38.605486 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.605390 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cd4678d6-jxznk" event={"ID":"666b226e-5777-43a7-b8a1-0bfbba372ed8","Type":"ContainerDied","Data":"74b0cc17ee5bc13b6447597a58bab838d6ca80e0f23e99130dd16fa7ab5944b4"} Jun 01 20:37:38.605486 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.605407 2559 scope.go:117] "RemoveContainer" containerID="ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa" Jun 01 20:37:38.614667 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.614640 2559 scope.go:117] "RemoveContainer" containerID="ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa" Jun 01 20:37:38.615154 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:37:38.615128 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa\": container with ID starting with ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa not found: ID does not exist" containerID="ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa" Jun 01 20:37:38.615252 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.615159 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa"} err="failed to get container status \"ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa\": rpc error: code = NotFound desc = could not find container \"ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa\": container with ID starting with ed1cbabb15e7b093c92aa5bdda5aae3718dea596b6aa638fa1156335df4ac3fa not found: ID does not exist" Jun 01 20:37:38.624831 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.624808 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79cd4678d6-jxznk"] Jun 01 20:37:38.628785 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:38.628764 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79cd4678d6-jxznk"] Jun 01 20:37:39.830819 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:39.830784 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="666b226e-5777-43a7-b8a1-0bfbba372ed8" path="/var/lib/kubelet/pods/666b226e-5777-43a7-b8a1-0bfbba372ed8/volumes" Jun 01 20:37:40.986571 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:40.986545 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:37:40.990677 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:37:40.990649 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5fcb55cc7d-2xs4z" Jun 01 20:38:11.585444 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:11.585369 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:38:11.587615 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:11.587594 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818c75c1-e3d8-4a90-89de-ade8164c6d4a-metrics-certs\") pod \"network-metrics-daemon-z6xn6\" (UID: \"818c75c1-e3d8-4a90-89de-ade8164c6d4a\") " pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:38:11.628921 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:11.628895 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gkzmp\"" Jun 01 20:38:11.637306 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:11.637284 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6xn6" Jun 01 20:38:11.757283 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:11.757253 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z6xn6"] Jun 01 20:38:11.760680 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:38:11.760648 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818c75c1_e3d8_4a90_89de_ade8164c6d4a.slice/crio-ec18b50d6973836fbef85e20356db2f56745cd825d66db4beada7cdf3f5eb5e8 WatchSource:0}: Error finding container ec18b50d6973836fbef85e20356db2f56745cd825d66db4beada7cdf3f5eb5e8: Status 404 returned error can't find the container with id ec18b50d6973836fbef85e20356db2f56745cd825d66db4beada7cdf3f5eb5e8 Jun 01 20:38:12.701796 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:12.701758 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z6xn6" event={"ID":"818c75c1-e3d8-4a90-89de-ade8164c6d4a","Type":"ContainerStarted","Data":"ec18b50d6973836fbef85e20356db2f56745cd825d66db4beada7cdf3f5eb5e8"} Jun 01 20:38:13.705524 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:13.705487 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z6xn6" event={"ID":"818c75c1-e3d8-4a90-89de-ade8164c6d4a","Type":"ContainerStarted","Data":"c1ec6f486af8b6ce0a5e7eb2b6e270a1a17f15a4afdec5bdafa0561b3ac90a92"} Jun 01 20:38:13.705524 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:13.705526 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z6xn6" event={"ID":"818c75c1-e3d8-4a90-89de-ade8164c6d4a","Type":"ContainerStarted","Data":"09d2afef8f55f40d1cdb7ac2d3e15d21e578ea0959bfbee7ed75fa4fe3566b3e"} Jun 01 20:38:13.718733 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:13.718683 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-z6xn6" podStartSLOduration=253.768568942 podStartE2EDuration="4m14.718668451s" podCreationTimestamp="2026-06-01 20:33:59 +0000 UTC" firstStartedPulling="2026-06-01 20:38:11.762466906 +0000 UTC m=+252.469839425" lastFinishedPulling="2026-06-01 20:38:12.712566419 +0000 UTC m=+253.419938934" observedRunningTime="2026-06-01 20:38:13.71812056 +0000 UTC m=+254.425493098" watchObservedRunningTime="2026-06-01 20:38:13.718668451 +0000 UTC m=+254.426040993" Jun 01 20:38:20.710415 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.710320 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn"] Jun 01 20:38:20.710836 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.710614 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="666b226e-5777-43a7-b8a1-0bfbba372ed8" containerName="console" Jun 01 20:38:20.710836 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.710627 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="666b226e-5777-43a7-b8a1-0bfbba372ed8" containerName="console" Jun 01 20:38:20.710836 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.710682 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="666b226e-5777-43a7-b8a1-0bfbba372ed8" containerName="console" Jun 01 20:38:20.713086 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.713063 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.715040 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.715016 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 01 20:38:20.715040 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.715031 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 01 20:38:20.715249 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.715045 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-z5pbr\"" Jun 01 20:38:20.715249 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.715016 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 01 20:38:20.715249 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.715094 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 01 20:38:20.715569 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.715552 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 01 20:38:20.719612 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.719594 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 01 20:38:20.728486 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.728456 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn"] Jun 01 20:38:20.859292 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.859254 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xkwp\" (UniqueName: \"kubernetes.io/projected/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-kube-api-access-8xkwp\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.859292 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.859292 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.859528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.859315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-metrics-client-ca\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.859528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.859345 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-serving-certs-ca-bundle\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.859528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.859405 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-telemeter-client-tls\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.859528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.859480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-secret-telemeter-client\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.859528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.859511 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.859528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.859528 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-federate-client-tls\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.959988 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.959951 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.959988 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.959991 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-metrics-client-ca\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.960233 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.960018 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-serving-certs-ca-bundle\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.960233 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.960077 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-telemeter-client-tls\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.960233 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.960155 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-secret-telemeter-client\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.960233 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.960202 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.960233 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.960228 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-federate-client-tls\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.960518 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.960270 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8xkwp\" (UniqueName: \"kubernetes.io/projected/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-kube-api-access-8xkwp\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.960821 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.960765 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-serving-certs-ca-bundle\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.961095 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.961072 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.961548 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.961526 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-metrics-client-ca\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.962786 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.962742 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-secret-telemeter-client\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.962882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.962838 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.962882 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.962839 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-telemeter-client-tls\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.963377 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.963356 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-federate-client-tls\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:20.967522 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:20.967502 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xkwp\" (UniqueName: \"kubernetes.io/projected/3c13d5ee-d8a8-4a4c-a1f8-663979899dcc-kube-api-access-8xkwp\") pod \"telemeter-client-6d7b8b5d4b-sw9rn\" (UID: \"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc\") " pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:21.023053 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:21.023018 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" Jun 01 20:38:21.166735 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:21.166711 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn"] Jun 01 20:38:21.169523 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:38:21.169487 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c13d5ee_d8a8_4a4c_a1f8_663979899dcc.slice/crio-4ba7a623bca757e0859e83b3cfc2b8bebcca3d4a77b5a962a54f38809ec2b626 WatchSource:0}: Error finding container 4ba7a623bca757e0859e83b3cfc2b8bebcca3d4a77b5a962a54f38809ec2b626: Status 404 returned error can't find the container with id 4ba7a623bca757e0859e83b3cfc2b8bebcca3d4a77b5a962a54f38809ec2b626 Jun 01 20:38:21.732725 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:21.732680 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" event={"ID":"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc","Type":"ContainerStarted","Data":"4ba7a623bca757e0859e83b3cfc2b8bebcca3d4a77b5a962a54f38809ec2b626"} Jun 01 20:38:24.743478 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:24.743405 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" event={"ID":"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc","Type":"ContainerStarted","Data":"1b3fe8b55d192b115cc7bb9a529f6ae80b0466ad72bdcb69db9f3506df191690"} Jun 01 20:38:24.743478 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:24.743480 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" event={"ID":"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc","Type":"ContainerStarted","Data":"704393cdecd96eb06f7243e31090d202ec0015480676933ceeaf0b954669e603"} Jun 01 20:38:24.743877 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:24.743490 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" event={"ID":"3c13d5ee-d8a8-4a4c-a1f8-663979899dcc","Type":"ContainerStarted","Data":"ec35ce39438efd5b4c18c3bb4e2fd83b8765562e53804fdaf6387c7812a1407e"} Jun 01 20:38:24.761568 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:24.761524 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d7b8b5d4b-sw9rn" podStartSLOduration=1.969753726 podStartE2EDuration="4.761511466s" podCreationTimestamp="2026-06-01 20:38:20 +0000 UTC" firstStartedPulling="2026-06-01 20:38:21.171768015 +0000 UTC m=+261.879140534" lastFinishedPulling="2026-06-01 20:38:23.963525757 +0000 UTC m=+264.670898274" observedRunningTime="2026-06-01 20:38:24.759753884 +0000 UTC m=+265.467126404" watchObservedRunningTime="2026-06-01 20:38:24.761511466 +0000 UTC m=+265.468884003" Jun 01 20:38:59.706219 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:38:59.706190 2559 kubelet.go:1628] "Image garbage collection succeeded" Jun 01 20:41:41.506729 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.506693 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs"] Jun 01 20:41:41.509827 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.509804 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" Jun 01 20:41:41.511743 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.511706 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-kqc2b\"" Jun 01 20:41:41.511853 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.511748 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 01 20:41:41.511853 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.511754 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 01 20:41:41.520713 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.520694 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs"] Jun 01 20:41:41.559513 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.559485 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfd9729f-44ea-43e7-b127-c6b61739fd48-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-xklhs\" (UID: \"dfd9729f-44ea-43e7-b127-c6b61739fd48\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" Jun 01 20:41:41.559655 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.559528 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9dlx\" (UniqueName: \"kubernetes.io/projected/dfd9729f-44ea-43e7-b127-c6b61739fd48-kube-api-access-r9dlx\") pod \"cert-manager-operator-controller-manager-54b9655956-xklhs\" (UID: \"dfd9729f-44ea-43e7-b127-c6b61739fd48\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" Jun 01 20:41:41.660629 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.660591 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfd9729f-44ea-43e7-b127-c6b61739fd48-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-xklhs\" (UID: \"dfd9729f-44ea-43e7-b127-c6b61739fd48\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" Jun 01 20:41:41.660785 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.660635 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r9dlx\" (UniqueName: \"kubernetes.io/projected/dfd9729f-44ea-43e7-b127-c6b61739fd48-kube-api-access-r9dlx\") pod \"cert-manager-operator-controller-manager-54b9655956-xklhs\" (UID: \"dfd9729f-44ea-43e7-b127-c6b61739fd48\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" Jun 01 20:41:41.661003 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.660982 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfd9729f-44ea-43e7-b127-c6b61739fd48-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-xklhs\" (UID: \"dfd9729f-44ea-43e7-b127-c6b61739fd48\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" Jun 01 20:41:41.668258 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.668235 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9dlx\" (UniqueName: \"kubernetes.io/projected/dfd9729f-44ea-43e7-b127-c6b61739fd48-kube-api-access-r9dlx\") pod \"cert-manager-operator-controller-manager-54b9655956-xklhs\" (UID: \"dfd9729f-44ea-43e7-b127-c6b61739fd48\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" Jun 01 20:41:41.819473 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.819364 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" Jun 01 20:41:41.951477 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.951449 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs"] Jun 01 20:41:41.955508 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:41:41.955473 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd9729f_44ea_43e7_b127_c6b61739fd48.slice/crio-85ac6f7194d8cfd23be6fcdc78a0a1059e3ee91e56b8942aef9c2d2133183c95 WatchSource:0}: Error finding container 85ac6f7194d8cfd23be6fcdc78a0a1059e3ee91e56b8942aef9c2d2133183c95: Status 404 returned error can't find the container with id 85ac6f7194d8cfd23be6fcdc78a0a1059e3ee91e56b8942aef9c2d2133183c95 Jun 01 20:41:41.957864 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:41.957847 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 01 20:41:42.261369 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:42.261331 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" event={"ID":"dfd9729f-44ea-43e7-b127-c6b61739fd48","Type":"ContainerStarted","Data":"85ac6f7194d8cfd23be6fcdc78a0a1059e3ee91e56b8942aef9c2d2133183c95"} Jun 01 20:41:44.269181 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:44.269094 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" event={"ID":"dfd9729f-44ea-43e7-b127-c6b61739fd48","Type":"ContainerStarted","Data":"1aacb385b6d7ca79aabd8b5914082511025083461e1a78155f2b5ba46e0014b8"} Jun 01 20:41:44.286911 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:41:44.286858 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-xklhs" podStartSLOduration=1.291510993 podStartE2EDuration="3.286844672s" podCreationTimestamp="2026-06-01 20:41:41 +0000 UTC" firstStartedPulling="2026-06-01 20:41:41.957977577 +0000 UTC m=+462.665350092" lastFinishedPulling="2026-06-01 20:41:43.953311257 +0000 UTC m=+464.660683771" observedRunningTime="2026-06-01 20:41:44.286174049 +0000 UTC m=+464.993546606" watchObservedRunningTime="2026-06-01 20:41:44.286844672 +0000 UTC m=+464.994217208" Jun 01 20:42:03.393894 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.393854 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns"] Jun 01 20:42:03.396964 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.396946 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" Jun 01 20:42:03.398797 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.398777 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 01 20:42:03.398969 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.398951 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-977wl\"" Jun 01 20:42:03.399018 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.398960 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 01 20:42:03.405865 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.405844 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns"] Jun 01 20:42:03.531353 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.531317 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87579\" (UniqueName: \"kubernetes.io/projected/9e982334-2307-4c5b-b722-0233e8ee4a78-kube-api-access-87579\") pod \"openshift-lws-operator-bfc7f696d-9l7ns\" (UID: \"9e982334-2307-4c5b-b722-0233e8ee4a78\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" Jun 01 20:42:03.531538 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.531364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e982334-2307-4c5b-b722-0233e8ee4a78-tmp\") pod \"openshift-lws-operator-bfc7f696d-9l7ns\" (UID: \"9e982334-2307-4c5b-b722-0233e8ee4a78\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" Jun 01 20:42:03.632482 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.632413 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-87579\" (UniqueName: \"kubernetes.io/projected/9e982334-2307-4c5b-b722-0233e8ee4a78-kube-api-access-87579\") pod \"openshift-lws-operator-bfc7f696d-9l7ns\" (UID: \"9e982334-2307-4c5b-b722-0233e8ee4a78\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" Jun 01 20:42:03.632482 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.632487 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e982334-2307-4c5b-b722-0233e8ee4a78-tmp\") pod \"openshift-lws-operator-bfc7f696d-9l7ns\" (UID: \"9e982334-2307-4c5b-b722-0233e8ee4a78\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" Jun 01 20:42:03.632852 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.632836 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e982334-2307-4c5b-b722-0233e8ee4a78-tmp\") pod \"openshift-lws-operator-bfc7f696d-9l7ns\" (UID: \"9e982334-2307-4c5b-b722-0233e8ee4a78\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" Jun 01 20:42:03.639784 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.639760 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87579\" (UniqueName: \"kubernetes.io/projected/9e982334-2307-4c5b-b722-0233e8ee4a78-kube-api-access-87579\") pod \"openshift-lws-operator-bfc7f696d-9l7ns\" (UID: \"9e982334-2307-4c5b-b722-0233e8ee4a78\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" Jun 01 20:42:03.706246 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.706165 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" Jun 01 20:42:03.831044 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:42:03.831014 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e982334_2307_4c5b_b722_0233e8ee4a78.slice/crio-3cb3a0f8954bc659696f6830ac08d7ab8c61e7bcb819010da70e9df021fdee4c WatchSource:0}: Error finding container 3cb3a0f8954bc659696f6830ac08d7ab8c61e7bcb819010da70e9df021fdee4c: Status 404 returned error can't find the container with id 3cb3a0f8954bc659696f6830ac08d7ab8c61e7bcb819010da70e9df021fdee4c Jun 01 20:42:03.831226 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:03.831206 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns"] Jun 01 20:42:04.328155 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:04.328123 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" event={"ID":"9e982334-2307-4c5b-b722-0233e8ee4a78","Type":"ContainerStarted","Data":"3cb3a0f8954bc659696f6830ac08d7ab8c61e7bcb819010da70e9df021fdee4c"} Jun 01 20:42:08.344249 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:08.344204 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" event={"ID":"9e982334-2307-4c5b-b722-0233e8ee4a78","Type":"ContainerStarted","Data":"47573fa9777fc552b090d5e679c51eb2534959372ba2aa036461ba4b97266b22"} Jun 01 20:42:08.359770 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:08.359715 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-9l7ns" podStartSLOduration=1.5329720610000002 podStartE2EDuration="5.359701225s" podCreationTimestamp="2026-06-01 20:42:03 +0000 UTC" firstStartedPulling="2026-06-01 20:42:03.832511869 +0000 UTC m=+484.539884384" lastFinishedPulling="2026-06-01 20:42:07.65924103 +0000 UTC m=+488.366613548" observedRunningTime="2026-06-01 20:42:08.357542244 +0000 UTC m=+489.064914780" watchObservedRunningTime="2026-06-01 20:42:08.359701225 +0000 UTC m=+489.067073762" Jun 01 20:42:15.321957 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.321927 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s"] Jun 01 20:42:15.325257 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.325238 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.327923 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.327897 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-thwds\"" Jun 01 20:42:15.327923 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.327897 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 01 20:42:15.328066 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.327903 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 01 20:42:15.328066 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.327959 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 01 20:42:15.333239 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.333211 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s"] Jun 01 20:42:15.431790 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.431759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdjqg\" (UniqueName: \"kubernetes.io/projected/d18633f5-00a1-460b-bd4f-be239aa3025f-kube-api-access-fdjqg\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.431790 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.431800 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d18633f5-00a1-460b-bd4f-be239aa3025f-metrics-cert\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.431993 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.431839 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d18633f5-00a1-460b-bd4f-be239aa3025f-manager-config\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.431993 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.431867 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d18633f5-00a1-460b-bd4f-be239aa3025f-cert\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.532502 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.532466 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d18633f5-00a1-460b-bd4f-be239aa3025f-manager-config\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.532682 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.532514 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d18633f5-00a1-460b-bd4f-be239aa3025f-cert\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.532682 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.532574 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fdjqg\" (UniqueName: \"kubernetes.io/projected/d18633f5-00a1-460b-bd4f-be239aa3025f-kube-api-access-fdjqg\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.532682 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.532599 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d18633f5-00a1-460b-bd4f-be239aa3025f-metrics-cert\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.533162 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.533140 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d18633f5-00a1-460b-bd4f-be239aa3025f-manager-config\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.535134 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.535113 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d18633f5-00a1-460b-bd4f-be239aa3025f-metrics-cert\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.535190 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.535116 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d18633f5-00a1-460b-bd4f-be239aa3025f-cert\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.539724 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.539698 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdjqg\" (UniqueName: \"kubernetes.io/projected/d18633f5-00a1-460b-bd4f-be239aa3025f-kube-api-access-fdjqg\") pod \"lws-controller-manager-58fff47c47-4r49s\" (UID: \"d18633f5-00a1-460b-bd4f-be239aa3025f\") " pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.634563 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.634498 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:15.756293 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:15.756257 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s"] Jun 01 20:42:15.759337 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:42:15.759312 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18633f5_00a1_460b_bd4f_be239aa3025f.slice/crio-cabb1836e358ddc0b888f1876723adce507681b19e250991bca180e6f44078d7 WatchSource:0}: Error finding container cabb1836e358ddc0b888f1876723adce507681b19e250991bca180e6f44078d7: Status 404 returned error can't find the container with id cabb1836e358ddc0b888f1876723adce507681b19e250991bca180e6f44078d7 Jun 01 20:42:16.367826 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:16.367783 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" event={"ID":"d18633f5-00a1-460b-bd4f-be239aa3025f","Type":"ContainerStarted","Data":"cabb1836e358ddc0b888f1876723adce507681b19e250991bca180e6f44078d7"} Jun 01 20:42:19.378051 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:19.378016 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" event={"ID":"d18633f5-00a1-460b-bd4f-be239aa3025f","Type":"ContainerStarted","Data":"c146856bfd13b39360854b68351f5b2ff3773637475240023eac2cf10cfe50d5"} Jun 01 20:42:19.378464 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:19.378126 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:19.393255 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:19.393194 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" podStartSLOduration=1.585923215 podStartE2EDuration="4.393177943s" podCreationTimestamp="2026-06-01 20:42:15 +0000 UTC" firstStartedPulling="2026-06-01 20:42:15.761046716 +0000 UTC m=+496.468419231" lastFinishedPulling="2026-06-01 20:42:18.568301444 +0000 UTC m=+499.275673959" observedRunningTime="2026-06-01 20:42:19.391194724 +0000 UTC m=+500.098567261" watchObservedRunningTime="2026-06-01 20:42:19.393177943 +0000 UTC m=+500.100550482" Jun 01 20:42:23.591534 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.591501 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5"] Jun 01 20:42:23.595222 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.595194 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.597683 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.597657 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 01 20:42:23.597823 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.597657 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zv4vq\"" Jun 01 20:42:23.597823 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.597693 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 01 20:42:23.597823 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.597664 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 01 20:42:23.598033 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.598020 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 01 20:42:23.606548 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.606523 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5"] Jun 01 20:42:23.709277 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.709237 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/28f3dd61-4614-4e31-b334-974983fa1b2c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.709277 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.709275 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rnmn\" (UniqueName: \"kubernetes.io/projected/28f3dd61-4614-4e31-b334-974983fa1b2c-kube-api-access-4rnmn\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.709526 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.709306 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/28f3dd61-4614-4e31-b334-974983fa1b2c-manifests\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.709526 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.709362 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28f3dd61-4614-4e31-b334-974983fa1b2c-webhook-cert\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.709526 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.709390 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28f3dd61-4614-4e31-b334-974983fa1b2c-tmp\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.810399 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.810363 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/28f3dd61-4614-4e31-b334-974983fa1b2c-manifests\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.810693 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.810409 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28f3dd61-4614-4e31-b334-974983fa1b2c-webhook-cert\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.810693 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.810455 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28f3dd61-4614-4e31-b334-974983fa1b2c-tmp\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.810693 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.810506 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/28f3dd61-4614-4e31-b334-974983fa1b2c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.810693 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.810627 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4rnmn\" (UniqueName: \"kubernetes.io/projected/28f3dd61-4614-4e31-b334-974983fa1b2c-kube-api-access-4rnmn\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.810921 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.810774 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/28f3dd61-4614-4e31-b334-974983fa1b2c-manifests\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.812813 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.812791 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28f3dd61-4614-4e31-b334-974983fa1b2c-tmp\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.812944 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.812925 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28f3dd61-4614-4e31-b334-974983fa1b2c-webhook-cert\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.813045 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.813024 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/28f3dd61-4614-4e31-b334-974983fa1b2c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.818644 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.818614 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rnmn\" (UniqueName: \"kubernetes.io/projected/28f3dd61-4614-4e31-b334-974983fa1b2c-kube-api-access-4rnmn\") pod \"opendatahub-operator-controller-manager-64857648b-8g9x5\" (UID: \"28f3dd61-4614-4e31-b334-974983fa1b2c\") " pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:23.905888 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:23.905854 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:24.038869 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:24.038784 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5"] Jun 01 20:42:24.042138 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:42:24.042111 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f3dd61_4614_4e31_b334_974983fa1b2c.slice/crio-721c6f41fafe51c4be37abd74dcf1f3e1f756d225aee0f5207ee45a8663948ac WatchSource:0}: Error finding container 721c6f41fafe51c4be37abd74dcf1f3e1f756d225aee0f5207ee45a8663948ac: Status 404 returned error can't find the container with id 721c6f41fafe51c4be37abd74dcf1f3e1f756d225aee0f5207ee45a8663948ac Jun 01 20:42:24.392613 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:24.392576 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" event={"ID":"28f3dd61-4614-4e31-b334-974983fa1b2c","Type":"ContainerStarted","Data":"721c6f41fafe51c4be37abd74dcf1f3e1f756d225aee0f5207ee45a8663948ac"} Jun 01 20:42:27.402931 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:27.402852 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" event={"ID":"28f3dd61-4614-4e31-b334-974983fa1b2c","Type":"ContainerStarted","Data":"16e4c60134ebb7a08ce57f0de2871596cb7199ede929319e61ba44cf0b63611e"} Jun 01 20:42:28.407158 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:28.407122 2559 generic.go:358] "Generic (PLEG): container finished" podID="28f3dd61-4614-4e31-b334-974983fa1b2c" containerID="16e4c60134ebb7a08ce57f0de2871596cb7199ede929319e61ba44cf0b63611e" exitCode=0 Jun 01 20:42:28.407551 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:28.407164 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" event={"ID":"28f3dd61-4614-4e31-b334-974983fa1b2c","Type":"ContainerDied","Data":"16e4c60134ebb7a08ce57f0de2871596cb7199ede929319e61ba44cf0b63611e"} Jun 01 20:42:29.411812 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:29.411778 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" event={"ID":"28f3dd61-4614-4e31-b334-974983fa1b2c","Type":"ContainerStarted","Data":"293a734b3cff0b5df2fdc7c5386737251b8548161a585d4dd9906f18e69b4e69"} Jun 01 20:42:29.412250 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:29.411879 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:29.432225 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:29.432175 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" podStartSLOduration=1.679071209 podStartE2EDuration="6.432156875s" podCreationTimestamp="2026-06-01 20:42:23 +0000 UTC" firstStartedPulling="2026-06-01 20:42:24.044002205 +0000 UTC m=+504.751374723" lastFinishedPulling="2026-06-01 20:42:28.797087859 +0000 UTC m=+509.504460389" observedRunningTime="2026-06-01 20:42:29.430818984 +0000 UTC m=+510.138191521" watchObservedRunningTime="2026-06-01 20:42:29.432156875 +0000 UTC m=+510.139529411" Jun 01 20:42:30.383295 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:30.383265 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58fff47c47-4r49s" Jun 01 20:42:40.417153 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:40.417121 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64857648b-8g9x5" Jun 01 20:42:46.033886 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.033847 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs"] Jun 01 20:42:46.049691 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.049657 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.050106 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.050074 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs"] Jun 01 20:42:46.053228 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.053203 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 01 20:42:46.053684 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.053659 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 01 20:42:46.053900 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.053876 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 01 20:42:46.053965 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.053935 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cpxv9\"" Jun 01 20:42:46.054440 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.052416 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 01 20:42:46.089000 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.088969 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0807bb2a-960e-4504-bcba-10190766f686-tls-certs\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.089140 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.089005 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78p8v\" (UniqueName: \"kubernetes.io/projected/0807bb2a-960e-4504-bcba-10190766f686-kube-api-access-78p8v\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.089140 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.089071 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0807bb2a-960e-4504-bcba-10190766f686-tmp\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.190321 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.190279 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-78p8v\" (UniqueName: \"kubernetes.io/projected/0807bb2a-960e-4504-bcba-10190766f686-kube-api-access-78p8v\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.190494 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.190444 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0807bb2a-960e-4504-bcba-10190766f686-tmp\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.190494 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.190485 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0807bb2a-960e-4504-bcba-10190766f686-tls-certs\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.190624 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:42:46.190582 2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 01 20:42:46.190667 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:42:46.190636 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0807bb2a-960e-4504-bcba-10190766f686-tls-certs podName:0807bb2a-960e-4504-bcba-10190766f686 nodeName:}" failed. No retries permitted until 2026-06-01 20:42:46.69062106 +0000 UTC m=+527.397993574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0807bb2a-960e-4504-bcba-10190766f686-tls-certs") pod "kube-auth-proxy-6855b9fc46-4jczs" (UID: "0807bb2a-960e-4504-bcba-10190766f686") : secret "kube-auth-proxy-tls" not found Jun 01 20:42:46.192773 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.192748 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0807bb2a-960e-4504-bcba-10190766f686-tmp\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.207939 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.207907 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78p8v\" (UniqueName: \"kubernetes.io/projected/0807bb2a-960e-4504-bcba-10190766f686-kube-api-access-78p8v\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.695135 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.695092 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0807bb2a-960e-4504-bcba-10190766f686-tls-certs\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.697613 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.697579 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0807bb2a-960e-4504-bcba-10190766f686-tls-certs\") pod \"kube-auth-proxy-6855b9fc46-4jczs\" (UID: \"0807bb2a-960e-4504-bcba-10190766f686\") " pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:46.961980 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:46.961887 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" Jun 01 20:42:47.086897 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:47.086868 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs"] Jun 01 20:42:47.089089 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:42:47.089060 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0807bb2a_960e_4504_bcba_10190766f686.slice/crio-855f14127817c793aef0681e5529da5adf746484e5e02dd83635909601f823a0 WatchSource:0}: Error finding container 855f14127817c793aef0681e5529da5adf746484e5e02dd83635909601f823a0: Status 404 returned error can't find the container with id 855f14127817c793aef0681e5529da5adf746484e5e02dd83635909601f823a0 Jun 01 20:42:47.469331 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:47.469296 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" event={"ID":"0807bb2a-960e-4504-bcba-10190766f686","Type":"ContainerStarted","Data":"855f14127817c793aef0681e5529da5adf746484e5e02dd83635909601f823a0"} Jun 01 20:42:50.478985 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:50.478945 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" event={"ID":"0807bb2a-960e-4504-bcba-10190766f686","Type":"ContainerStarted","Data":"0259e62ab00d9cdf7a9ad5b0220d2cd4d7061c47edf4f530f7595eeb8c328169"} Jun 01 20:42:50.495642 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:42:50.495592 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6855b9fc46-4jczs" podStartSLOduration=1.279188383 podStartE2EDuration="4.495576417s" podCreationTimestamp="2026-06-01 20:42:46 +0000 UTC" firstStartedPulling="2026-06-01 20:42:47.090781294 +0000 UTC m=+527.798153812" lastFinishedPulling="2026-06-01 20:42:50.307169317 +0000 UTC m=+531.014541846" observedRunningTime="2026-06-01 20:42:50.493702851 +0000 UTC m=+531.201075388" watchObservedRunningTime="2026-06-01 20:42:50.495576417 +0000 UTC m=+531.202949036" Jun 01 20:44:37.481114 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.481031 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q"] Jun 01 20:44:37.484164 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.484148 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:37.490990 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.490967 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 01 20:44:37.491104 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.490971 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 01 20:44:37.491517 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.491494 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 01 20:44:37.491619 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.491560 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 01 20:44:37.491689 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.491498 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9gh6b\"" Jun 01 20:44:37.505193 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.505166 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q"] Jun 01 20:44:37.641627 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.641592 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvpcd\" (UniqueName: \"kubernetes.io/projected/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-kube-api-access-qvpcd\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:37.641829 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.641653 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:37.641829 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.641767 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:37.743080 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.742991 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:37.743080 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.743049 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:37.743080 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.743080 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qvpcd\" (UniqueName: \"kubernetes.io/projected/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-kube-api-access-qvpcd\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:37.743352 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:44:37.743139 2559 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 01 20:44:37.743352 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:44:37.743212 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-plugin-serving-cert podName:f239ad67-4e5d-47b0-a6dc-682e9d5eba23 nodeName:}" failed. No retries permitted until 2026-06-01 20:44:38.243195753 +0000 UTC m=+638.950568268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-nc62q" (UID: "f239ad67-4e5d-47b0-a6dc-682e9d5eba23") : secret "plugin-serving-cert" not found Jun 01 20:44:37.743813 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.743789 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:37.778191 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:37.778159 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvpcd\" (UniqueName: \"kubernetes.io/projected/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-kube-api-access-qvpcd\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:38.247441 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:38.247375 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:38.249838 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:38.249811 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f239ad67-4e5d-47b0-a6dc-682e9d5eba23-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-nc62q\" (UID: \"f239ad67-4e5d-47b0-a6dc-682e9d5eba23\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:38.393127 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:38.393092 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" Jun 01 20:44:38.525222 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:38.525193 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q"] Jun 01 20:44:38.527499 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:44:38.527468 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf239ad67_4e5d_47b0_a6dc_682e9d5eba23.slice/crio-f10d57a2bbcc3fd53d9151d4eb35ed4694f5c8d7e1333d2c61fbda571a55d7f8 WatchSource:0}: Error finding container f10d57a2bbcc3fd53d9151d4eb35ed4694f5c8d7e1333d2c61fbda571a55d7f8: Status 404 returned error can't find the container with id f10d57a2bbcc3fd53d9151d4eb35ed4694f5c8d7e1333d2c61fbda571a55d7f8 Jun 01 20:44:38.816557 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:44:38.816471 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" event={"ID":"f239ad67-4e5d-47b0-a6dc-682e9d5eba23","Type":"ContainerStarted","Data":"f10d57a2bbcc3fd53d9151d4eb35ed4694f5c8d7e1333d2c61fbda571a55d7f8"} Jun 01 20:45:03.907323 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:03.907288 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" event={"ID":"f239ad67-4e5d-47b0-a6dc-682e9d5eba23","Type":"ContainerStarted","Data":"fdf3ddb9a50fdf97a9617b37b9303bedf7e317d3470d261f62a8daaee2d3cb2c"} Jun 01 20:45:03.954283 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:03.954180 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-nc62q" podStartSLOduration=1.779350674 podStartE2EDuration="26.954164413s" podCreationTimestamp="2026-06-01 20:44:37 +0000 UTC" firstStartedPulling="2026-06-01 20:44:38.528782966 +0000 UTC m=+639.236155495" lastFinishedPulling="2026-06-01 20:45:03.70359672 +0000 UTC m=+664.410969234" observedRunningTime="2026-06-01 20:45:03.953156507 +0000 UTC m=+664.660529069" watchObservedRunningTime="2026-06-01 20:45:03.954164413 +0000 UTC m=+664.661536950" Jun 01 20:45:22.615494 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.615455 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-rjm49"] Jun 01 20:45:22.618639 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.618621 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" Jun 01 20:45:22.620615 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.620598 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-kj9bs\"" Jun 01 20:45:22.633264 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.630839 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-rjm49"] Jun 01 20:45:22.728370 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.728326 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj6w5\" (UniqueName: \"kubernetes.io/projected/50f99098-e738-4ee2-aff7-0c1ca286de50-kube-api-access-lj6w5\") pod \"authorino-f99f4b5cd-rjm49\" (UID: \"50f99098-e738-4ee2-aff7-0c1ca286de50\") " pod="kuadrant-system/authorino-f99f4b5cd-rjm49" Jun 01 20:45:22.778440 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.778388 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-96545"] Jun 01 20:45:22.781815 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.781797 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-96545" Jun 01 20:45:22.791058 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.791036 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-96545"] Jun 01 20:45:22.829655 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.829631 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lj6w5\" (UniqueName: \"kubernetes.io/projected/50f99098-e738-4ee2-aff7-0c1ca286de50-kube-api-access-lj6w5\") pod \"authorino-f99f4b5cd-rjm49\" (UID: \"50f99098-e738-4ee2-aff7-0c1ca286de50\") " pod="kuadrant-system/authorino-f99f4b5cd-rjm49" Jun 01 20:45:22.840778 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.840757 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj6w5\" (UniqueName: \"kubernetes.io/projected/50f99098-e738-4ee2-aff7-0c1ca286de50-kube-api-access-lj6w5\") pod \"authorino-f99f4b5cd-rjm49\" (UID: \"50f99098-e738-4ee2-aff7-0c1ca286de50\") " pod="kuadrant-system/authorino-f99f4b5cd-rjm49" Jun 01 20:45:22.928138 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.928108 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" Jun 01 20:45:22.930996 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:22.930975 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bdp7\" (UniqueName: \"kubernetes.io/projected/59c07ab3-ab94-4e6b-be64-68602bd9a00d-kube-api-access-5bdp7\") pod \"authorino-7498df8756-96545\" (UID: \"59c07ab3-ab94-4e6b-be64-68602bd9a00d\") " pod="kuadrant-system/authorino-7498df8756-96545" Jun 01 20:45:23.032022 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:23.031985 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5bdp7\" (UniqueName: \"kubernetes.io/projected/59c07ab3-ab94-4e6b-be64-68602bd9a00d-kube-api-access-5bdp7\") pod \"authorino-7498df8756-96545\" (UID: \"59c07ab3-ab94-4e6b-be64-68602bd9a00d\") " pod="kuadrant-system/authorino-7498df8756-96545" Jun 01 20:45:23.041095 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:23.041066 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bdp7\" (UniqueName: \"kubernetes.io/projected/59c07ab3-ab94-4e6b-be64-68602bd9a00d-kube-api-access-5bdp7\") pod \"authorino-7498df8756-96545\" (UID: \"59c07ab3-ab94-4e6b-be64-68602bd9a00d\") " pod="kuadrant-system/authorino-7498df8756-96545" Jun 01 20:45:23.052352 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:23.052331 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-rjm49"] Jun 01 20:45:23.054406 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:45:23.054375 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50f99098_e738_4ee2_aff7_0c1ca286de50.slice/crio-13a1d50654a531cc211dc563967dce3491f1bb76d3c81b0f5f36787e4043d55a WatchSource:0}: Error finding container 13a1d50654a531cc211dc563967dce3491f1bb76d3c81b0f5f36787e4043d55a: Status 404 returned error can't find the container with id 13a1d50654a531cc211dc563967dce3491f1bb76d3c81b0f5f36787e4043d55a Jun 01 20:45:23.091035 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:23.091008 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-96545" Jun 01 20:45:23.212556 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:23.212529 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-96545"] Jun 01 20:45:23.214795 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:45:23.214772 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59c07ab3_ab94_4e6b_be64_68602bd9a00d.slice/crio-9d75eb96a310093f8f24a49102dfb00fafe4c1711852d9d701f4141370bc0289 WatchSource:0}: Error finding container 9d75eb96a310093f8f24a49102dfb00fafe4c1711852d9d701f4141370bc0289: Status 404 returned error can't find the container with id 9d75eb96a310093f8f24a49102dfb00fafe4c1711852d9d701f4141370bc0289 Jun 01 20:45:23.973893 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:23.973849 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" event={"ID":"50f99098-e738-4ee2-aff7-0c1ca286de50","Type":"ContainerStarted","Data":"13a1d50654a531cc211dc563967dce3491f1bb76d3c81b0f5f36787e4043d55a"} Jun 01 20:45:23.976665 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:23.976627 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-96545" event={"ID":"59c07ab3-ab94-4e6b-be64-68602bd9a00d","Type":"ContainerStarted","Data":"9d75eb96a310093f8f24a49102dfb00fafe4c1711852d9d701f4141370bc0289"} Jun 01 20:45:26.988306 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:26.988272 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" event={"ID":"50f99098-e738-4ee2-aff7-0c1ca286de50","Type":"ContainerStarted","Data":"c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1"} Jun 01 20:45:26.989504 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:26.989478 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-96545" event={"ID":"59c07ab3-ab94-4e6b-be64-68602bd9a00d","Type":"ContainerStarted","Data":"9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59"} Jun 01 20:45:27.005386 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:27.005340 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" podStartSLOduration=1.528199315 podStartE2EDuration="5.005327595s" podCreationTimestamp="2026-06-01 20:45:22 +0000 UTC" firstStartedPulling="2026-06-01 20:45:23.055675437 +0000 UTC m=+683.763047956" lastFinishedPulling="2026-06-01 20:45:26.532803718 +0000 UTC m=+687.240176236" observedRunningTime="2026-06-01 20:45:27.00338234 +0000 UTC m=+687.710754878" watchObservedRunningTime="2026-06-01 20:45:27.005327595 +0000 UTC m=+687.712700131" Jun 01 20:45:27.018407 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:27.018357 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-96545" podStartSLOduration=1.691373852 podStartE2EDuration="5.018340506s" podCreationTimestamp="2026-06-01 20:45:22 +0000 UTC" firstStartedPulling="2026-06-01 20:45:23.216112134 +0000 UTC m=+683.923484654" lastFinishedPulling="2026-06-01 20:45:26.54307879 +0000 UTC m=+687.250451308" observedRunningTime="2026-06-01 20:45:27.016868113 +0000 UTC m=+687.724240649" watchObservedRunningTime="2026-06-01 20:45:27.018340506 +0000 UTC m=+687.725713045" Jun 01 20:45:27.047978 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:27.047945 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-rjm49"] Jun 01 20:45:28.995051 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:28.995008 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" podUID="50f99098-e738-4ee2-aff7-0c1ca286de50" containerName="authorino" containerID="cri-o://c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1" gracePeriod=30 Jun 01 20:45:29.237810 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:29.237785 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" Jun 01 20:45:29.389200 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:29.389102 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj6w5\" (UniqueName: \"kubernetes.io/projected/50f99098-e738-4ee2-aff7-0c1ca286de50-kube-api-access-lj6w5\") pod \"50f99098-e738-4ee2-aff7-0c1ca286de50\" (UID: \"50f99098-e738-4ee2-aff7-0c1ca286de50\") " Jun 01 20:45:29.391275 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:29.391248 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f99098-e738-4ee2-aff7-0c1ca286de50-kube-api-access-lj6w5" (OuterVolumeSpecName: "kube-api-access-lj6w5") pod "50f99098-e738-4ee2-aff7-0c1ca286de50" (UID: "50f99098-e738-4ee2-aff7-0c1ca286de50"). InnerVolumeSpecName "kube-api-access-lj6w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 20:45:29.490447 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:29.490389 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj6w5\" (UniqueName: \"kubernetes.io/projected/50f99098-e738-4ee2-aff7-0c1ca286de50-kube-api-access-lj6w5\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:45:29.998667 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:29.998634 2559 generic.go:358] "Generic (PLEG): container finished" podID="50f99098-e738-4ee2-aff7-0c1ca286de50" containerID="c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1" exitCode=0 Jun 01 20:45:29.999013 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:29.998729 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" event={"ID":"50f99098-e738-4ee2-aff7-0c1ca286de50","Type":"ContainerDied","Data":"c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1"} Jun 01 20:45:29.999013 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:29.998750 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" event={"ID":"50f99098-e738-4ee2-aff7-0c1ca286de50","Type":"ContainerDied","Data":"13a1d50654a531cc211dc563967dce3491f1bb76d3c81b0f5f36787e4043d55a"} Jun 01 20:45:29.999013 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:29.998765 2559 scope.go:117] "RemoveContainer" containerID="c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1" Jun 01 20:45:29.999013 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:29.998849 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-rjm49" Jun 01 20:45:30.006910 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:30.006891 2559 scope.go:117] "RemoveContainer" containerID="c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1" Jun 01 20:45:30.007161 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:45:30.007136 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1\": container with ID starting with c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1 not found: ID does not exist" containerID="c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1" Jun 01 20:45:30.007226 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:30.007168 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1"} err="failed to get container status \"c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1\": rpc error: code = NotFound desc = could not find container \"c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1\": container with ID starting with c52f3185d5f26a7443121ce5a953296b961de65299c4be18ded8dc7e3a2f34d1 not found: ID does not exist" Jun 01 20:45:30.012220 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:30.012198 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-rjm49"] Jun 01 20:45:30.018314 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:30.018293 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-rjm49"] Jun 01 20:45:31.831862 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:45:31.831822 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f99098-e738-4ee2-aff7-0c1ca286de50" path="/var/lib/kubelet/pods/50f99098-e738-4ee2-aff7-0c1ca286de50/volumes" Jun 01 20:46:17.578563 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.578531 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 01 20:46:17.579056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.578845 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f99098-e738-4ee2-aff7-0c1ca286de50" containerName="authorino" Jun 01 20:46:17.579056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.578857 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f99098-e738-4ee2-aff7-0c1ca286de50" containerName="authorino" Jun 01 20:46:17.579056 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.578906 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="50f99098-e738-4ee2-aff7-0c1ca286de50" containerName="authorino" Jun 01 20:46:17.582819 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.582803 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 01 20:46:17.584714 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.584692 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-8fpvg\"" Jun 01 20:46:17.584840 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.584745 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 01 20:46:17.585039 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.585022 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 01 20:46:17.585110 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.585038 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 01 20:46:17.589702 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.589667 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 01 20:46:17.687653 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.687616 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vht9x\" (UniqueName: \"kubernetes.io/projected/f36e8ea1-1967-4a04-a565-777a49ad07f4-kube-api-access-vht9x\") pod \"maas-keycloak-0\" (UID: \"f36e8ea1-1967-4a04-a565-777a49ad07f4\") " pod="keycloak-system/maas-keycloak-0" Jun 01 20:46:17.788839 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.788804 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vht9x\" (UniqueName: \"kubernetes.io/projected/f36e8ea1-1967-4a04-a565-777a49ad07f4-kube-api-access-vht9x\") pod \"maas-keycloak-0\" (UID: \"f36e8ea1-1967-4a04-a565-777a49ad07f4\") " pod="keycloak-system/maas-keycloak-0" Jun 01 20:46:17.799262 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.799235 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vht9x\" (UniqueName: \"kubernetes.io/projected/f36e8ea1-1967-4a04-a565-777a49ad07f4-kube-api-access-vht9x\") pod \"maas-keycloak-0\" (UID: \"f36e8ea1-1967-4a04-a565-777a49ad07f4\") " pod="keycloak-system/maas-keycloak-0" Jun 01 20:46:17.893514 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:17.893481 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 01 20:46:18.019905 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:18.019879 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 01 20:46:18.021835 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:46:18.021807 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36e8ea1_1967_4a04_a565_777a49ad07f4.slice/crio-77656d6b46d4a2730f958cb1a32860fc4f394170054ed3a023b9932d3bc90d32 WatchSource:0}: Error finding container 77656d6b46d4a2730f958cb1a32860fc4f394170054ed3a023b9932d3bc90d32: Status 404 returned error can't find the container with id 77656d6b46d4a2730f958cb1a32860fc4f394170054ed3a023b9932d3bc90d32 Jun 01 20:46:18.148071 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:18.147981 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f36e8ea1-1967-4a04-a565-777a49ad07f4","Type":"ContainerStarted","Data":"77656d6b46d4a2730f958cb1a32860fc4f394170054ed3a023b9932d3bc90d32"} Jun 01 20:46:23.173627 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:23.173594 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f36e8ea1-1967-4a04-a565-777a49ad07f4","Type":"ContainerStarted","Data":"a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f"} Jun 01 20:46:23.192054 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:23.192004 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.140544432 podStartE2EDuration="6.191989024s" podCreationTimestamp="2026-06-01 20:46:17 +0000 UTC" firstStartedPulling="2026-06-01 20:46:18.023092939 +0000 UTC m=+738.730465453" lastFinishedPulling="2026-06-01 20:46:23.074537517 +0000 UTC m=+743.781910045" observedRunningTime="2026-06-01 20:46:23.190275633 +0000 UTC m=+743.897648170" watchObservedRunningTime="2026-06-01 20:46:23.191989024 +0000 UTC m=+743.899361560" Jun 01 20:46:23.894096 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:23.894048 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 01 20:46:23.895918 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:23.895879 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:24.893919 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:24.893873 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:25.894915 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:25.894859 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:26.894797 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:26.894742 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:27.893893 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:27.893848 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 01 20:46:27.894509 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:27.894472 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:28.894792 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:28.894737 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:29.901451 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:29.897289 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:30.894757 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:30.894705 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:31.894601 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:31.894549 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:32.894567 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:32.894517 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:33.894165 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:33.894104 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:34.894171 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:34.894120 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:35.894585 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:35.894533 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.24:9000/health/started\": dial tcp 10.133.0.24:9000: connect: connection refused" Jun 01 20:46:37.025341 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:37.025305 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 01 20:46:37.045051 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:37.045001 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 01 20:46:47.032319 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:47.032283 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 01 20:46:47.907456 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:47.907407 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-2mxjv"] Jun 01 20:46:47.917512 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:47.917484 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-2mxjv"] Jun 01 20:46:47.917660 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:47.917585 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" Jun 01 20:46:47.962742 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:47.962706 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsv2n\" (UniqueName: \"kubernetes.io/projected/530ddb95-1ede-4c20-ad55-6bb88bbdad03-kube-api-access-qsv2n\") pod \"authorino-8b475cf9f-2mxjv\" (UID: \"530ddb95-1ede-4c20-ad55-6bb88bbdad03\") " pod="kuadrant-system/authorino-8b475cf9f-2mxjv" Jun 01 20:46:48.063731 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.063682 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qsv2n\" (UniqueName: \"kubernetes.io/projected/530ddb95-1ede-4c20-ad55-6bb88bbdad03-kube-api-access-qsv2n\") pod \"authorino-8b475cf9f-2mxjv\" (UID: \"530ddb95-1ede-4c20-ad55-6bb88bbdad03\") " pod="kuadrant-system/authorino-8b475cf9f-2mxjv" Jun 01 20:46:48.071975 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.071945 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsv2n\" (UniqueName: \"kubernetes.io/projected/530ddb95-1ede-4c20-ad55-6bb88bbdad03-kube-api-access-qsv2n\") pod \"authorino-8b475cf9f-2mxjv\" (UID: \"530ddb95-1ede-4c20-ad55-6bb88bbdad03\") " pod="kuadrant-system/authorino-8b475cf9f-2mxjv" Jun 01 20:46:48.138790 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.138753 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-2mxjv"] Jun 01 20:46:48.139002 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.138990 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" Jun 01 20:46:48.162329 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.162262 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bcd4c4c99-hlg4g"] Jun 01 20:46:48.166666 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.166644 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" Jun 01 20:46:48.172052 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.172002 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bcd4c4c99-hlg4g"] Jun 01 20:46:48.265952 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.265831 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-2mxjv"] Jun 01 20:46:48.266176 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.266155 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz56s\" (UniqueName: \"kubernetes.io/projected/6b9d0fd4-433b-4d30-8002-aecbfa997fc6-kube-api-access-jz56s\") pod \"authorino-bcd4c4c99-hlg4g\" (UID: \"6b9d0fd4-433b-4d30-8002-aecbfa997fc6\") " pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" Jun 01 20:46:48.268739 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:46:48.268707 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod530ddb95_1ede_4c20_ad55_6bb88bbdad03.slice/crio-d64069ac6f51bbad83a9643adb82e7a73b734f9e5ac2880a2d96c4ec13531a4b WatchSource:0}: Error finding container d64069ac6f51bbad83a9643adb82e7a73b734f9e5ac2880a2d96c4ec13531a4b: Status 404 returned error can't find the container with id d64069ac6f51bbad83a9643adb82e7a73b734f9e5ac2880a2d96c4ec13531a4b Jun 01 20:46:48.270010 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.269971 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 01 20:46:48.272952 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.272916 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" event={"ID":"530ddb95-1ede-4c20-ad55-6bb88bbdad03","Type":"ContainerStarted","Data":"d64069ac6f51bbad83a9643adb82e7a73b734f9e5ac2880a2d96c4ec13531a4b"} Jun 01 20:46:48.367135 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.367095 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jz56s\" (UniqueName: \"kubernetes.io/projected/6b9d0fd4-433b-4d30-8002-aecbfa997fc6-kube-api-access-jz56s\") pod \"authorino-bcd4c4c99-hlg4g\" (UID: \"6b9d0fd4-433b-4d30-8002-aecbfa997fc6\") " pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" Jun 01 20:46:48.375405 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.375377 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz56s\" (UniqueName: \"kubernetes.io/projected/6b9d0fd4-433b-4d30-8002-aecbfa997fc6-kube-api-access-jz56s\") pod \"authorino-bcd4c4c99-hlg4g\" (UID: \"6b9d0fd4-433b-4d30-8002-aecbfa997fc6\") " pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" Jun 01 20:46:48.477867 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.477779 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" Jun 01 20:46:48.599945 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:48.599920 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bcd4c4c99-hlg4g"] Jun 01 20:46:48.601842 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:46:48.601808 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b9d0fd4_433b_4d30_8002_aecbfa997fc6.slice/crio-1c485467cab390229468277f06ab9ef73344735759ae9b6ad9c424d63cbb7487 WatchSource:0}: Error finding container 1c485467cab390229468277f06ab9ef73344735759ae9b6ad9c424d63cbb7487: Status 404 returned error can't find the container with id 1c485467cab390229468277f06ab9ef73344735759ae9b6ad9c424d63cbb7487 Jun 01 20:46:49.276895 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.276796 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" event={"ID":"530ddb95-1ede-4c20-ad55-6bb88bbdad03","Type":"ContainerStarted","Data":"91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff"} Jun 01 20:46:49.276895 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.276803 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" podUID="530ddb95-1ede-4c20-ad55-6bb88bbdad03" containerName="authorino" containerID="cri-o://91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff" gracePeriod=30 Jun 01 20:46:49.278184 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.278154 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" event={"ID":"6b9d0fd4-433b-4d30-8002-aecbfa997fc6","Type":"ContainerStarted","Data":"74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd"} Jun 01 20:46:49.278282 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.278191 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" event={"ID":"6b9d0fd4-433b-4d30-8002-aecbfa997fc6","Type":"ContainerStarted","Data":"1c485467cab390229468277f06ab9ef73344735759ae9b6ad9c424d63cbb7487"} Jun 01 20:46:49.290875 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.290718 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" podStartSLOduration=1.93299404 podStartE2EDuration="2.290704083s" podCreationTimestamp="2026-06-01 20:46:47 +0000 UTC" firstStartedPulling="2026-06-01 20:46:48.270123758 +0000 UTC m=+768.977496273" lastFinishedPulling="2026-06-01 20:46:48.627833801 +0000 UTC m=+769.335206316" observedRunningTime="2026-06-01 20:46:49.289296043 +0000 UTC m=+769.996668580" watchObservedRunningTime="2026-06-01 20:46:49.290704083 +0000 UTC m=+769.998076619" Jun 01 20:46:49.303220 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.303172 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" podStartSLOduration=1.006402498 podStartE2EDuration="1.303159896s" podCreationTimestamp="2026-06-01 20:46:48 +0000 UTC" firstStartedPulling="2026-06-01 20:46:48.603117159 +0000 UTC m=+769.310489677" lastFinishedPulling="2026-06-01 20:46:48.899874558 +0000 UTC m=+769.607247075" observedRunningTime="2026-06-01 20:46:49.30138408 +0000 UTC m=+770.008756617" watchObservedRunningTime="2026-06-01 20:46:49.303159896 +0000 UTC m=+770.010532513" Jun 01 20:46:49.325146 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.325117 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-96545"] Jun 01 20:46:49.325355 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.325315 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-96545" podUID="59c07ab3-ab94-4e6b-be64-68602bd9a00d" containerName="authorino" containerID="cri-o://9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59" gracePeriod=30 Jun 01 20:46:49.453189 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.453157 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cffdb4dbc-5d9cc"] Jun 01 20:46:49.456443 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.456412 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:46:49.458244 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.458221 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 01 20:46:49.468238 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.468198 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cffdb4dbc-5d9cc"] Jun 01 20:46:49.475570 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.474945 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fvnx\" (UniqueName: \"kubernetes.io/projected/8379d005-e9b3-4523-b245-7828cc34edf6-kube-api-access-4fvnx\") pod \"authorino-5cffdb4dbc-5d9cc\" (UID: \"8379d005-e9b3-4523-b245-7828cc34edf6\") " pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:46:49.475570 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.475020 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8379d005-e9b3-4523-b245-7828cc34edf6-tls-cert\") pod \"authorino-5cffdb4dbc-5d9cc\" (UID: \"8379d005-e9b3-4523-b245-7828cc34edf6\") " pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:46:49.575810 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.575774 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8379d005-e9b3-4523-b245-7828cc34edf6-tls-cert\") pod \"authorino-5cffdb4dbc-5d9cc\" (UID: \"8379d005-e9b3-4523-b245-7828cc34edf6\") " pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:46:49.575946 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.575826 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4fvnx\" (UniqueName: \"kubernetes.io/projected/8379d005-e9b3-4523-b245-7828cc34edf6-kube-api-access-4fvnx\") pod \"authorino-5cffdb4dbc-5d9cc\" (UID: \"8379d005-e9b3-4523-b245-7828cc34edf6\") " pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:46:49.577185 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.577163 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" Jun 01 20:46:49.578205 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.578185 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8379d005-e9b3-4523-b245-7828cc34edf6-tls-cert\") pod \"authorino-5cffdb4dbc-5d9cc\" (UID: \"8379d005-e9b3-4523-b245-7828cc34edf6\") " pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:46:49.583002 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.582979 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fvnx\" (UniqueName: \"kubernetes.io/projected/8379d005-e9b3-4523-b245-7828cc34edf6-kube-api-access-4fvnx\") pod \"authorino-5cffdb4dbc-5d9cc\" (UID: \"8379d005-e9b3-4523-b245-7828cc34edf6\") " pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:46:49.595645 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.595619 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-96545" Jun 01 20:46:49.676694 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.676657 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsv2n\" (UniqueName: \"kubernetes.io/projected/530ddb95-1ede-4c20-ad55-6bb88bbdad03-kube-api-access-qsv2n\") pod \"530ddb95-1ede-4c20-ad55-6bb88bbdad03\" (UID: \"530ddb95-1ede-4c20-ad55-6bb88bbdad03\") " Jun 01 20:46:49.676694 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.676705 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bdp7\" (UniqueName: \"kubernetes.io/projected/59c07ab3-ab94-4e6b-be64-68602bd9a00d-kube-api-access-5bdp7\") pod \"59c07ab3-ab94-4e6b-be64-68602bd9a00d\" (UID: \"59c07ab3-ab94-4e6b-be64-68602bd9a00d\") " Jun 01 20:46:49.678722 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.678690 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c07ab3-ab94-4e6b-be64-68602bd9a00d-kube-api-access-5bdp7" (OuterVolumeSpecName: "kube-api-access-5bdp7") pod "59c07ab3-ab94-4e6b-be64-68602bd9a00d" (UID: "59c07ab3-ab94-4e6b-be64-68602bd9a00d"). InnerVolumeSpecName "kube-api-access-5bdp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 20:46:49.678722 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.678718 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/530ddb95-1ede-4c20-ad55-6bb88bbdad03-kube-api-access-qsv2n" (OuterVolumeSpecName: "kube-api-access-qsv2n") pod "530ddb95-1ede-4c20-ad55-6bb88bbdad03" (UID: "530ddb95-1ede-4c20-ad55-6bb88bbdad03"). InnerVolumeSpecName "kube-api-access-qsv2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 20:46:49.768114 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.768074 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:46:49.778131 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.778080 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsv2n\" (UniqueName: \"kubernetes.io/projected/530ddb95-1ede-4c20-ad55-6bb88bbdad03-kube-api-access-qsv2n\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:46:49.778131 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.778104 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bdp7\" (UniqueName: \"kubernetes.io/projected/59c07ab3-ab94-4e6b-be64-68602bd9a00d-kube-api-access-5bdp7\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:46:49.897371 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:49.897346 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cffdb4dbc-5d9cc"] Jun 01 20:46:49.899803 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:46:49.899778 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8379d005_e9b3_4523_b245_7828cc34edf6.slice/crio-ff9cbb6d798855e7ec049dc50f2cc1ebf52e7530d5ba1a27fb7afd9664fe723c WatchSource:0}: Error finding container ff9cbb6d798855e7ec049dc50f2cc1ebf52e7530d5ba1a27fb7afd9664fe723c: Status 404 returned error can't find the container with id ff9cbb6d798855e7ec049dc50f2cc1ebf52e7530d5ba1a27fb7afd9664fe723c Jun 01 20:46:50.283075 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.283041 2559 generic.go:358] "Generic (PLEG): container finished" podID="59c07ab3-ab94-4e6b-be64-68602bd9a00d" containerID="9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59" exitCode=0 Jun 01 20:46:50.283496 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.283093 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-96545" Jun 01 20:46:50.283496 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.283121 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-96545" event={"ID":"59c07ab3-ab94-4e6b-be64-68602bd9a00d","Type":"ContainerDied","Data":"9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59"} Jun 01 20:46:50.283496 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.283159 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-96545" event={"ID":"59c07ab3-ab94-4e6b-be64-68602bd9a00d","Type":"ContainerDied","Data":"9d75eb96a310093f8f24a49102dfb00fafe4c1711852d9d701f4141370bc0289"} Jun 01 20:46:50.283496 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.283193 2559 scope.go:117] "RemoveContainer" containerID="9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59" Jun 01 20:46:50.284439 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.284287 2559 generic.go:358] "Generic (PLEG): container finished" podID="530ddb95-1ede-4c20-ad55-6bb88bbdad03" containerID="91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff" exitCode=0 Jun 01 20:46:50.284439 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.284326 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" Jun 01 20:46:50.284439 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.284370 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" event={"ID":"530ddb95-1ede-4c20-ad55-6bb88bbdad03","Type":"ContainerDied","Data":"91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff"} Jun 01 20:46:50.284439 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.284403 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-2mxjv" event={"ID":"530ddb95-1ede-4c20-ad55-6bb88bbdad03","Type":"ContainerDied","Data":"d64069ac6f51bbad83a9643adb82e7a73b734f9e5ac2880a2d96c4ec13531a4b"} Jun 01 20:46:50.285739 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.285715 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" event={"ID":"8379d005-e9b3-4523-b245-7828cc34edf6","Type":"ContainerStarted","Data":"ff9cbb6d798855e7ec049dc50f2cc1ebf52e7530d5ba1a27fb7afd9664fe723c"} Jun 01 20:46:50.297389 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.297373 2559 scope.go:117] "RemoveContainer" containerID="9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59" Jun 01 20:46:50.297641 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:46:50.297624 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59\": container with ID starting with 9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59 not found: ID does not exist" containerID="9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59" Jun 01 20:46:50.297693 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.297648 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59"} err="failed to get container status \"9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59\": rpc error: code = NotFound desc = could not find container \"9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59\": container with ID starting with 9600d1c79af6f7e8f7b3e4f792fb4898d9d987beb02c21396566849d45ae2a59 not found: ID does not exist" Jun 01 20:46:50.297693 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.297667 2559 scope.go:117] "RemoveContainer" containerID="91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff" Jun 01 20:46:50.316497 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.316475 2559 scope.go:117] "RemoveContainer" containerID="91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff" Jun 01 20:46:50.316789 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:46:50.316770 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff\": container with ID starting with 91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff not found: ID does not exist" containerID="91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff" Jun 01 20:46:50.316841 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.316804 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff"} err="failed to get container status \"91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff\": rpc error: code = NotFound desc = could not find container \"91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff\": container with ID starting with 91feb171f54288bcdff233830d03350a8a566acdea2873e8b2f19045a64841ff not found: ID does not exist" Jun 01 20:46:50.324091 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.324050 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-96545"] Jun 01 20:46:50.326962 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.326939 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-96545"] Jun 01 20:46:50.336270 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.336247 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-2mxjv"] Jun 01 20:46:50.342672 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:50.342649 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-2mxjv"] Jun 01 20:46:51.291696 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.291651 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" event={"ID":"8379d005-e9b3-4523-b245-7828cc34edf6","Type":"ContainerStarted","Data":"091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147"} Jun 01 20:46:51.308583 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.308532 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" podStartSLOduration=1.9430927919999998 podStartE2EDuration="2.308517053s" podCreationTimestamp="2026-06-01 20:46:49 +0000 UTC" firstStartedPulling="2026-06-01 20:46:49.900990438 +0000 UTC m=+770.608362956" lastFinishedPulling="2026-06-01 20:46:50.266414692 +0000 UTC m=+770.973787217" observedRunningTime="2026-06-01 20:46:51.306080316 +0000 UTC m=+772.013452853" watchObservedRunningTime="2026-06-01 20:46:51.308517053 +0000 UTC m=+772.015889590" Jun 01 20:46:51.330180 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.330139 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bcd4c4c99-hlg4g"] Jun 01 20:46:51.330414 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.330386 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" podUID="6b9d0fd4-433b-4d30-8002-aecbfa997fc6" containerName="authorino" containerID="cri-o://74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd" gracePeriod=30 Jun 01 20:46:51.574801 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.574779 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" Jun 01 20:46:51.696284 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.696248 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz56s\" (UniqueName: \"kubernetes.io/projected/6b9d0fd4-433b-4d30-8002-aecbfa997fc6-kube-api-access-jz56s\") pod \"6b9d0fd4-433b-4d30-8002-aecbfa997fc6\" (UID: \"6b9d0fd4-433b-4d30-8002-aecbfa997fc6\") " Jun 01 20:46:51.698329 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.698302 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9d0fd4-433b-4d30-8002-aecbfa997fc6-kube-api-access-jz56s" (OuterVolumeSpecName: "kube-api-access-jz56s") pod "6b9d0fd4-433b-4d30-8002-aecbfa997fc6" (UID: "6b9d0fd4-433b-4d30-8002-aecbfa997fc6"). InnerVolumeSpecName "kube-api-access-jz56s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 20:46:51.797541 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.797488 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz56s\" (UniqueName: \"kubernetes.io/projected/6b9d0fd4-433b-4d30-8002-aecbfa997fc6-kube-api-access-jz56s\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:46:51.833063 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.832983 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="530ddb95-1ede-4c20-ad55-6bb88bbdad03" path="/var/lib/kubelet/pods/530ddb95-1ede-4c20-ad55-6bb88bbdad03/volumes" Jun 01 20:46:51.833309 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:51.833296 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c07ab3-ab94-4e6b-be64-68602bd9a00d" path="/var/lib/kubelet/pods/59c07ab3-ab94-4e6b-be64-68602bd9a00d/volumes" Jun 01 20:46:52.295432 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.295382 2559 generic.go:358] "Generic (PLEG): container finished" podID="6b9d0fd4-433b-4d30-8002-aecbfa997fc6" containerID="74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd" exitCode=0 Jun 01 20:46:52.295918 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.295451 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" Jun 01 20:46:52.295918 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.295470 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" event={"ID":"6b9d0fd4-433b-4d30-8002-aecbfa997fc6","Type":"ContainerDied","Data":"74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd"} Jun 01 20:46:52.295918 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.295510 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bcd4c4c99-hlg4g" event={"ID":"6b9d0fd4-433b-4d30-8002-aecbfa997fc6","Type":"ContainerDied","Data":"1c485467cab390229468277f06ab9ef73344735759ae9b6ad9c424d63cbb7487"} Jun 01 20:46:52.295918 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.295531 2559 scope.go:117] "RemoveContainer" containerID="74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd" Jun 01 20:46:52.303509 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.303407 2559 scope.go:117] "RemoveContainer" containerID="74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd" Jun 01 20:46:52.303686 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:46:52.303667 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd\": container with ID starting with 74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd not found: ID does not exist" containerID="74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd" Jun 01 20:46:52.303736 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.303700 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd"} err="failed to get container status \"74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd\": rpc error: code = NotFound desc = could not find container \"74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd\": container with ID starting with 74815afdfc6fdb555f62a091fbab31b9227a1a5f194206c1b6679d46038b99cd not found: ID does not exist" Jun 01 20:46:52.311225 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.311202 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bcd4c4c99-hlg4g"] Jun 01 20:46:52.312954 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.312934 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-bcd4c4c99-hlg4g"] Jun 01 20:46:52.631165 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631087 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-844986557d-c9sd5"] Jun 01 20:46:52.631390 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631377 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c07ab3-ab94-4e6b-be64-68602bd9a00d" containerName="authorino" Jun 01 20:46:52.631455 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631392 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c07ab3-ab94-4e6b-be64-68602bd9a00d" containerName="authorino" Jun 01 20:46:52.631455 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631407 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="530ddb95-1ede-4c20-ad55-6bb88bbdad03" containerName="authorino" Jun 01 20:46:52.631455 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631412 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="530ddb95-1ede-4c20-ad55-6bb88bbdad03" containerName="authorino" Jun 01 20:46:52.631455 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631438 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b9d0fd4-433b-4d30-8002-aecbfa997fc6" containerName="authorino" Jun 01 20:46:52.631455 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631446 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b9d0fd4-433b-4d30-8002-aecbfa997fc6" containerName="authorino" Jun 01 20:46:52.631594 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631513 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="59c07ab3-ab94-4e6b-be64-68602bd9a00d" containerName="authorino" Jun 01 20:46:52.631594 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631523 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="530ddb95-1ede-4c20-ad55-6bb88bbdad03" containerName="authorino" Jun 01 20:46:52.631594 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.631530 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="6b9d0fd4-433b-4d30-8002-aecbfa997fc6" containerName="authorino" Jun 01 20:46:52.635548 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.635526 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:46:52.637489 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.637469 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qqzxc\"" Jun 01 20:46:52.637585 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.637482 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 01 20:46:52.637585 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.637476 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 01 20:46:52.645446 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.645403 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-844986557d-c9sd5"] Jun 01 20:46:52.706540 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.706501 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c97efd61-b3fc-45b1-aee3-311ea77ff839-webhook-cert\") pod \"maas-controller-844986557d-c9sd5\" (UID: \"c97efd61-b3fc-45b1-aee3-311ea77ff839\") " pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:46:52.706689 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.706593 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt278\" (UniqueName: \"kubernetes.io/projected/c97efd61-b3fc-45b1-aee3-311ea77ff839-kube-api-access-gt278\") pod \"maas-controller-844986557d-c9sd5\" (UID: \"c97efd61-b3fc-45b1-aee3-311ea77ff839\") " pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:46:52.807829 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.807790 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gt278\" (UniqueName: \"kubernetes.io/projected/c97efd61-b3fc-45b1-aee3-311ea77ff839-kube-api-access-gt278\") pod \"maas-controller-844986557d-c9sd5\" (UID: \"c97efd61-b3fc-45b1-aee3-311ea77ff839\") " pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:46:52.808034 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.807871 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c97efd61-b3fc-45b1-aee3-311ea77ff839-webhook-cert\") pod \"maas-controller-844986557d-c9sd5\" (UID: \"c97efd61-b3fc-45b1-aee3-311ea77ff839\") " pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:46:52.810497 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.810472 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c97efd61-b3fc-45b1-aee3-311ea77ff839-webhook-cert\") pod \"maas-controller-844986557d-c9sd5\" (UID: \"c97efd61-b3fc-45b1-aee3-311ea77ff839\") " pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:46:52.811326 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.811304 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6745f5f64f-tpdqh"] Jun 01 20:46:52.814522 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.814507 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:46:52.815307 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.815285 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt278\" (UniqueName: \"kubernetes.io/projected/c97efd61-b3fc-45b1-aee3-311ea77ff839-kube-api-access-gt278\") pod \"maas-controller-844986557d-c9sd5\" (UID: \"c97efd61-b3fc-45b1-aee3-311ea77ff839\") " pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:46:52.822898 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.822874 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6745f5f64f-tpdqh"] Jun 01 20:46:52.908388 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.908352 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5858cfb8-74b6-4f11-817d-90e17f3d385c-webhook-cert\") pod \"maas-controller-6745f5f64f-tpdqh\" (UID: \"5858cfb8-74b6-4f11-817d-90e17f3d385c\") " pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:46:52.908597 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.908471 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95cnv\" (UniqueName: \"kubernetes.io/projected/5858cfb8-74b6-4f11-817d-90e17f3d385c-kube-api-access-95cnv\") pod \"maas-controller-6745f5f64f-tpdqh\" (UID: \"5858cfb8-74b6-4f11-817d-90e17f3d385c\") " pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:46:52.946416 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:52.946384 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:46:53.009808 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.009777 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5858cfb8-74b6-4f11-817d-90e17f3d385c-webhook-cert\") pod \"maas-controller-6745f5f64f-tpdqh\" (UID: \"5858cfb8-74b6-4f11-817d-90e17f3d385c\") " pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:46:53.009935 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.009850 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-95cnv\" (UniqueName: \"kubernetes.io/projected/5858cfb8-74b6-4f11-817d-90e17f3d385c-kube-api-access-95cnv\") pod \"maas-controller-6745f5f64f-tpdqh\" (UID: \"5858cfb8-74b6-4f11-817d-90e17f3d385c\") " pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:46:53.012088 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.012060 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5858cfb8-74b6-4f11-817d-90e17f3d385c-webhook-cert\") pod \"maas-controller-6745f5f64f-tpdqh\" (UID: \"5858cfb8-74b6-4f11-817d-90e17f3d385c\") " pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:46:53.021170 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.021146 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95cnv\" (UniqueName: \"kubernetes.io/projected/5858cfb8-74b6-4f11-817d-90e17f3d385c-kube-api-access-95cnv\") pod \"maas-controller-6745f5f64f-tpdqh\" (UID: \"5858cfb8-74b6-4f11-817d-90e17f3d385c\") " pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:46:53.072336 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.072303 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-844986557d-c9sd5"] Jun 01 20:46:53.091978 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:46:53.091949 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc97efd61_b3fc_45b1_aee3_311ea77ff839.slice/crio-6098175de2e3209e061e1d2c8d203af9bdb729a42bd20157754e412bf8c7f945 WatchSource:0}: Error finding container 6098175de2e3209e061e1d2c8d203af9bdb729a42bd20157754e412bf8c7f945: Status 404 returned error can't find the container with id 6098175de2e3209e061e1d2c8d203af9bdb729a42bd20157754e412bf8c7f945 Jun 01 20:46:53.134273 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.134233 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:46:53.269315 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.269290 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6745f5f64f-tpdqh"] Jun 01 20:46:53.270914 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:46:53.270885 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5858cfb8_74b6_4f11_817d_90e17f3d385c.slice/crio-95f007b5cd43db174e44f30ca9d64b4f31a92d3233ea47ac0775bf0ddeedfc6e WatchSource:0}: Error finding container 95f007b5cd43db174e44f30ca9d64b4f31a92d3233ea47ac0775bf0ddeedfc6e: Status 404 returned error can't find the container with id 95f007b5cd43db174e44f30ca9d64b4f31a92d3233ea47ac0775bf0ddeedfc6e Jun 01 20:46:53.301111 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.301081 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844986557d-c9sd5" event={"ID":"c97efd61-b3fc-45b1-aee3-311ea77ff839","Type":"ContainerStarted","Data":"6098175de2e3209e061e1d2c8d203af9bdb729a42bd20157754e412bf8c7f945"} Jun 01 20:46:53.302015 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.301989 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6745f5f64f-tpdqh" event={"ID":"5858cfb8-74b6-4f11-817d-90e17f3d385c","Type":"ContainerStarted","Data":"95f007b5cd43db174e44f30ca9d64b4f31a92d3233ea47ac0775bf0ddeedfc6e"} Jun 01 20:46:53.842126 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:53.842085 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b9d0fd4-433b-4d30-8002-aecbfa997fc6" path="/var/lib/kubelet/pods/6b9d0fd4-433b-4d30-8002-aecbfa997fc6/volumes" Jun 01 20:46:56.317526 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:56.317485 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844986557d-c9sd5" event={"ID":"c97efd61-b3fc-45b1-aee3-311ea77ff839","Type":"ContainerStarted","Data":"e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60"} Jun 01 20:46:56.318057 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:56.317544 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:46:56.318966 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:56.318936 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6745f5f64f-tpdqh" event={"ID":"5858cfb8-74b6-4f11-817d-90e17f3d385c","Type":"ContainerStarted","Data":"d5cc803d647f42ed47076d848eea3f61758c3a83c4eecc71eac546a7a81fd03a"} Jun 01 20:46:56.319080 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:56.319063 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:46:56.335018 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:56.334970 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-844986557d-c9sd5" podStartSLOduration=1.71977322 podStartE2EDuration="4.334958318s" podCreationTimestamp="2026-06-01 20:46:52 +0000 UTC" firstStartedPulling="2026-06-01 20:46:53.093181663 +0000 UTC m=+773.800554177" lastFinishedPulling="2026-06-01 20:46:55.70836676 +0000 UTC m=+776.415739275" observedRunningTime="2026-06-01 20:46:56.334238728 +0000 UTC m=+777.041611266" watchObservedRunningTime="2026-06-01 20:46:56.334958318 +0000 UTC m=+777.042330855" Jun 01 20:46:56.356881 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:46:56.356814 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6745f5f64f-tpdqh" podStartSLOduration=1.910759272 podStartE2EDuration="4.356804009s" podCreationTimestamp="2026-06-01 20:46:52 +0000 UTC" firstStartedPulling="2026-06-01 20:46:53.272204131 +0000 UTC m=+773.979576646" lastFinishedPulling="2026-06-01 20:46:55.718248858 +0000 UTC m=+776.425621383" observedRunningTime="2026-06-01 20:46:56.355625847 +0000 UTC m=+777.062998377" watchObservedRunningTime="2026-06-01 20:46:56.356804009 +0000 UTC m=+777.064176546" Jun 01 20:47:07.327996 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.327954 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:47:07.328411 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.328354 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6745f5f64f-tpdqh" Jun 01 20:47:07.382248 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.382211 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-844986557d-c9sd5"] Jun 01 20:47:07.382460 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.382434 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-844986557d-c9sd5" podUID="c97efd61-b3fc-45b1-aee3-311ea77ff839" containerName="manager" containerID="cri-o://e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60" gracePeriod=30 Jun 01 20:47:07.620745 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.620722 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:47:07.736592 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.736559 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c97efd61-b3fc-45b1-aee3-311ea77ff839-webhook-cert\") pod \"c97efd61-b3fc-45b1-aee3-311ea77ff839\" (UID: \"c97efd61-b3fc-45b1-aee3-311ea77ff839\") " Jun 01 20:47:07.736754 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.736627 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt278\" (UniqueName: \"kubernetes.io/projected/c97efd61-b3fc-45b1-aee3-311ea77ff839-kube-api-access-gt278\") pod \"c97efd61-b3fc-45b1-aee3-311ea77ff839\" (UID: \"c97efd61-b3fc-45b1-aee3-311ea77ff839\") " Jun 01 20:47:07.738672 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.738638 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c97efd61-b3fc-45b1-aee3-311ea77ff839-kube-api-access-gt278" (OuterVolumeSpecName: "kube-api-access-gt278") pod "c97efd61-b3fc-45b1-aee3-311ea77ff839" (UID: "c97efd61-b3fc-45b1-aee3-311ea77ff839"). InnerVolumeSpecName "kube-api-access-gt278". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 20:47:07.738839 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.738720 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c97efd61-b3fc-45b1-aee3-311ea77ff839-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c97efd61-b3fc-45b1-aee3-311ea77ff839" (UID: "c97efd61-b3fc-45b1-aee3-311ea77ff839"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 01 20:47:07.837109 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.837082 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c97efd61-b3fc-45b1-aee3-311ea77ff839-webhook-cert\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:47:07.837109 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:07.837109 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt278\" (UniqueName: \"kubernetes.io/projected/c97efd61-b3fc-45b1-aee3-311ea77ff839-kube-api-access-gt278\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:47:08.359469 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:08.359404 2559 generic.go:358] "Generic (PLEG): container finished" podID="c97efd61-b3fc-45b1-aee3-311ea77ff839" containerID="e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60" exitCode=0 Jun 01 20:47:08.359890 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:08.359492 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844986557d-c9sd5" Jun 01 20:47:08.359890 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:08.359501 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844986557d-c9sd5" event={"ID":"c97efd61-b3fc-45b1-aee3-311ea77ff839","Type":"ContainerDied","Data":"e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60"} Jun 01 20:47:08.359890 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:08.359544 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844986557d-c9sd5" event={"ID":"c97efd61-b3fc-45b1-aee3-311ea77ff839","Type":"ContainerDied","Data":"6098175de2e3209e061e1d2c8d203af9bdb729a42bd20157754e412bf8c7f945"} Jun 01 20:47:08.359890 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:08.359560 2559 scope.go:117] "RemoveContainer" containerID="e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60" Jun 01 20:47:08.367266 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:08.367246 2559 scope.go:117] "RemoveContainer" containerID="e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60" Jun 01 20:47:08.367518 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:47:08.367495 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60\": container with ID starting with e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60 not found: ID does not exist" containerID="e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60" Jun 01 20:47:08.367595 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:08.367526 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60"} err="failed to get container status \"e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60\": rpc error: code = NotFound desc = could not find container \"e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60\": container with ID starting with e99095fdb74948f15820ed20cd2a3bd8c92cc4c9ed9029341ba40499d6fa1f60 not found: ID does not exist" Jun 01 20:47:08.374237 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:08.374214 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-844986557d-c9sd5"] Jun 01 20:47:08.377514 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:08.377493 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-844986557d-c9sd5"] Jun 01 20:47:09.832643 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:09.832610 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c97efd61-b3fc-45b1-aee3-311ea77ff839" path="/var/lib/kubelet/pods/c97efd61-b3fc-45b1-aee3-311ea77ff839/volumes" Jun 01 20:47:26.751106 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.751068 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-75cfcc4598-mvlsw"] Jun 01 20:47:26.751563 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.751395 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c97efd61-b3fc-45b1-aee3-311ea77ff839" containerName="manager" Jun 01 20:47:26.751563 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.751407 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c97efd61-b3fc-45b1-aee3-311ea77ff839" containerName="manager" Jun 01 20:47:26.751563 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.751484 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="c97efd61-b3fc-45b1-aee3-311ea77ff839" containerName="manager" Jun 01 20:47:26.754370 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.754352 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:26.756414 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.756397 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 01 20:47:26.763699 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.763671 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-75cfcc4598-mvlsw"] Jun 01 20:47:26.898795 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.898761 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjr26\" (UniqueName: \"kubernetes.io/projected/999de3e3-d690-4ca4-a18c-18be3495bd18-kube-api-access-vjr26\") pod \"maas-api-75cfcc4598-mvlsw\" (UID: \"999de3e3-d690-4ca4-a18c-18be3495bd18\") " pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:26.898966 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.898821 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999de3e3-d690-4ca4-a18c-18be3495bd18-maas-api-tls\") pod \"maas-api-75cfcc4598-mvlsw\" (UID: \"999de3e3-d690-4ca4-a18c-18be3495bd18\") " pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:26.999736 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.999699 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vjr26\" (UniqueName: \"kubernetes.io/projected/999de3e3-d690-4ca4-a18c-18be3495bd18-kube-api-access-vjr26\") pod \"maas-api-75cfcc4598-mvlsw\" (UID: \"999de3e3-d690-4ca4-a18c-18be3495bd18\") " pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:26.999890 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:26.999764 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999de3e3-d690-4ca4-a18c-18be3495bd18-maas-api-tls\") pod \"maas-api-75cfcc4598-mvlsw\" (UID: \"999de3e3-d690-4ca4-a18c-18be3495bd18\") " pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:26.999890 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:47:26.999876 2559 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 01 20:47:26.999974 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:47:26.999939 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/999de3e3-d690-4ca4-a18c-18be3495bd18-maas-api-tls podName:999de3e3-d690-4ca4-a18c-18be3495bd18 nodeName:}" failed. No retries permitted until 2026-06-01 20:47:27.499922486 +0000 UTC m=+808.207295002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/999de3e3-d690-4ca4-a18c-18be3495bd18-maas-api-tls") pod "maas-api-75cfcc4598-mvlsw" (UID: "999de3e3-d690-4ca4-a18c-18be3495bd18") : secret "maas-api-serving-cert" not found Jun 01 20:47:27.008200 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:27.008133 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjr26\" (UniqueName: \"kubernetes.io/projected/999de3e3-d690-4ca4-a18c-18be3495bd18-kube-api-access-vjr26\") pod \"maas-api-75cfcc4598-mvlsw\" (UID: \"999de3e3-d690-4ca4-a18c-18be3495bd18\") " pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:27.505348 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:27.505312 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999de3e3-d690-4ca4-a18c-18be3495bd18-maas-api-tls\") pod \"maas-api-75cfcc4598-mvlsw\" (UID: \"999de3e3-d690-4ca4-a18c-18be3495bd18\") " pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:27.507777 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:27.507751 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999de3e3-d690-4ca4-a18c-18be3495bd18-maas-api-tls\") pod \"maas-api-75cfcc4598-mvlsw\" (UID: \"999de3e3-d690-4ca4-a18c-18be3495bd18\") " pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:27.665490 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:27.665454 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:27.791977 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:27.791949 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-75cfcc4598-mvlsw"] Jun 01 20:47:27.794408 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:47:27.794377 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999de3e3_d690_4ca4_a18c_18be3495bd18.slice/crio-358b8ce3537669055fd3249a82d0c7062b8af60ebbf984b70981e218e6237f24 WatchSource:0}: Error finding container 358b8ce3537669055fd3249a82d0c7062b8af60ebbf984b70981e218e6237f24: Status 404 returned error can't find the container with id 358b8ce3537669055fd3249a82d0c7062b8af60ebbf984b70981e218e6237f24 Jun 01 20:47:28.429377 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:28.429340 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-75cfcc4598-mvlsw" event={"ID":"999de3e3-d690-4ca4-a18c-18be3495bd18","Type":"ContainerStarted","Data":"358b8ce3537669055fd3249a82d0c7062b8af60ebbf984b70981e218e6237f24"} Jun 01 20:47:31.441110 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:31.441071 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-75cfcc4598-mvlsw" event={"ID":"999de3e3-d690-4ca4-a18c-18be3495bd18","Type":"ContainerStarted","Data":"13c9b1aede5bacd6b6e2efc86db1367a446d011b87864c63b29cf4874c933dfc"} Jun 01 20:47:31.441534 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:31.441219 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:31.454915 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:31.454873 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-75cfcc4598-mvlsw" podStartSLOduration=2.650659234 podStartE2EDuration="5.454860592s" podCreationTimestamp="2026-06-01 20:47:26 +0000 UTC" firstStartedPulling="2026-06-01 20:47:27.79561888 +0000 UTC m=+808.502991412" lastFinishedPulling="2026-06-01 20:47:30.59982024 +0000 UTC m=+811.307192770" observedRunningTime="2026-06-01 20:47:31.454496525 +0000 UTC m=+812.161869063" watchObservedRunningTime="2026-06-01 20:47:31.454860592 +0000 UTC m=+812.162233128" Jun 01 20:47:37.450582 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:37.450505 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-75cfcc4598-mvlsw" Jun 01 20:47:39.273243 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:39.273181 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 01 20:47:39.273732 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:39.273590 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" containerID="cri-o://a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f" gracePeriod=30 Jun 01 20:47:41.318354 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.318331 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.400846 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.400812 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vht9x\" (UniqueName: \"kubernetes.io/projected/f36e8ea1-1967-4a04-a565-777a49ad07f4-kube-api-access-vht9x\") pod \"f36e8ea1-1967-4a04-a565-777a49ad07f4\" (UID: \"f36e8ea1-1967-4a04-a565-777a49ad07f4\") " Jun 01 20:47:41.402937 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.402898 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36e8ea1-1967-4a04-a565-777a49ad07f4-kube-api-access-vht9x" (OuterVolumeSpecName: "kube-api-access-vht9x") pod "f36e8ea1-1967-4a04-a565-777a49ad07f4" (UID: "f36e8ea1-1967-4a04-a565-777a49ad07f4"). InnerVolumeSpecName "kube-api-access-vht9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 20:47:41.473632 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.473548 2559 generic.go:358] "Generic (PLEG): container finished" podID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerID="a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f" exitCode=143 Jun 01 20:47:41.473632 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.473607 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.473808 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.473624 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f36e8ea1-1967-4a04-a565-777a49ad07f4","Type":"ContainerDied","Data":"a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f"} Jun 01 20:47:41.473808 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.473664 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f36e8ea1-1967-4a04-a565-777a49ad07f4","Type":"ContainerDied","Data":"77656d6b46d4a2730f958cb1a32860fc4f394170054ed3a023b9932d3bc90d32"} Jun 01 20:47:41.473808 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.473680 2559 scope.go:117] "RemoveContainer" containerID="a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f" Jun 01 20:47:41.483045 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.483025 2559 scope.go:117] "RemoveContainer" containerID="a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f" Jun 01 20:47:41.483319 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:47:41.483300 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f\": container with ID starting with a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f not found: ID does not exist" containerID="a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f" Jun 01 20:47:41.483385 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.483333 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f"} err="failed to get container status \"a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f\": rpc error: code = NotFound desc = could not find container \"a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f\": container with ID starting with a28ee94aa9f149edcdb806ccd6c4c8ea1717bfb049ae15d888a828508c93527f not found: ID does not exist" Jun 01 20:47:41.493641 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.493617 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 01 20:47:41.496796 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.496773 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 01 20:47:41.501454 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.501414 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vht9x\" (UniqueName: \"kubernetes.io/projected/f36e8ea1-1967-4a04-a565-777a49ad07f4-kube-api-access-vht9x\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:47:41.516651 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.516623 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 01 20:47:41.516995 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.516982 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" Jun 01 20:47:41.517040 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.516998 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" Jun 01 20:47:41.517070 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.517058 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" containerName="keycloak" Jun 01 20:47:41.521223 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.521205 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.523391 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.523365 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 01 20:47:41.523391 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.523387 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 01 20:47:41.523560 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.523518 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-8fpvg\"" Jun 01 20:47:41.523623 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.523591 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 01 20:47:41.523749 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.523733 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 01 20:47:41.529577 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.529554 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 01 20:47:41.601916 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.601877 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4592\" (UniqueName: \"kubernetes.io/projected/872fa32a-d19b-4fb3-b13a-bf6af47e0e0e-kube-api-access-h4592\") pod \"maas-keycloak-0\" (UID: \"872fa32a-d19b-4fb3-b13a-bf6af47e0e0e\") " pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.601916 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.601920 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/872fa32a-d19b-4fb3-b13a-bf6af47e0e0e-test-realms\") pod \"maas-keycloak-0\" (UID: \"872fa32a-d19b-4fb3-b13a-bf6af47e0e0e\") " pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.702633 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.702599 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h4592\" (UniqueName: \"kubernetes.io/projected/872fa32a-d19b-4fb3-b13a-bf6af47e0e0e-kube-api-access-h4592\") pod \"maas-keycloak-0\" (UID: \"872fa32a-d19b-4fb3-b13a-bf6af47e0e0e\") " pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.702793 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.702641 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/872fa32a-d19b-4fb3-b13a-bf6af47e0e0e-test-realms\") pod \"maas-keycloak-0\" (UID: \"872fa32a-d19b-4fb3-b13a-bf6af47e0e0e\") " pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.703286 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.703268 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/872fa32a-d19b-4fb3-b13a-bf6af47e0e0e-test-realms\") pod \"maas-keycloak-0\" (UID: \"872fa32a-d19b-4fb3-b13a-bf6af47e0e0e\") " pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.710709 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.710679 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4592\" (UniqueName: \"kubernetes.io/projected/872fa32a-d19b-4fb3-b13a-bf6af47e0e0e-kube-api-access-h4592\") pod \"maas-keycloak-0\" (UID: \"872fa32a-d19b-4fb3-b13a-bf6af47e0e0e\") " pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.831858 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.831779 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f36e8ea1-1967-4a04-a565-777a49ad07f4" path="/var/lib/kubelet/pods/f36e8ea1-1967-4a04-a565-777a49ad07f4/volumes" Jun 01 20:47:41.832263 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.832238 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:41.981291 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:41.981266 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 01 20:47:41.983644 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:47:41.983607 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod872fa32a_d19b_4fb3_b13a_bf6af47e0e0e.slice/crio-9d65554579fda7fab3cc5127e71ce1828bc7242d56bea979efd09422d66baea6 WatchSource:0}: Error finding container 9d65554579fda7fab3cc5127e71ce1828bc7242d56bea979efd09422d66baea6: Status 404 returned error can't find the container with id 9d65554579fda7fab3cc5127e71ce1828bc7242d56bea979efd09422d66baea6 Jun 01 20:47:42.480093 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:42.480032 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"872fa32a-d19b-4fb3-b13a-bf6af47e0e0e","Type":"ContainerStarted","Data":"74a03cc427443d07352d6c573ff0779af64a0e0dee737ce819acb5ac8e7c5934"} Jun 01 20:47:42.480093 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:42.480075 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"872fa32a-d19b-4fb3-b13a-bf6af47e0e0e","Type":"ContainerStarted","Data":"9d65554579fda7fab3cc5127e71ce1828bc7242d56bea979efd09422d66baea6"} Jun 01 20:47:42.497680 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:42.497632 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.209783128 podStartE2EDuration="1.497618262s" podCreationTimestamp="2026-06-01 20:47:41 +0000 UTC" firstStartedPulling="2026-06-01 20:47:41.985091511 +0000 UTC m=+822.692464026" lastFinishedPulling="2026-06-01 20:47:42.272926642 +0000 UTC m=+822.980299160" observedRunningTime="2026-06-01 20:47:42.496365766 +0000 UTC m=+823.203738303" watchObservedRunningTime="2026-06-01 20:47:42.497618262 +0000 UTC m=+823.204990798" Jun 01 20:47:42.832749 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:42.832710 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:42.834443 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:42.834396 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:43.832748 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:43.832701 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:44.833343 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:44.833287 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:45.833055 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:45.833011 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:46.833665 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:46.833605 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:47.834026 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:47.833982 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:48.833050 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:48.832987 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:49.832912 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:49.832865 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:50.833261 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:50.833211 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:51.832810 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:51.832754 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:51.833237 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:51.833205 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:52.833277 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:52.833220 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:53.832959 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:53.832911 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:54.832861 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:54.832818 2559 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.31:9000/health/started\": dial tcp 10.133.0.31:9000: connect: connection refused" Jun 01 20:47:55.966587 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:55.966546 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 01 20:47:55.987028 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:47:55.986971 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="872fa32a-d19b-4fb3-b13a-bf6af47e0e0e" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 01 20:48:05.973528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:05.973490 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 01 20:48:16.674385 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.674347 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67dccf5fdf-p88d5"] Jun 01 20:48:16.679968 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.679948 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.681957 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.681937 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 01 20:48:16.685322 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.685297 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67dccf5fdf-p88d5"] Jun 01 20:48:16.825181 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.825145 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtdq4\" (UniqueName: \"kubernetes.io/projected/16011125-3547-45a6-856c-dbc3e809e74e-kube-api-access-jtdq4\") pod \"authorino-67dccf5fdf-p88d5\" (UID: \"16011125-3547-45a6-856c-dbc3e809e74e\") " pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.825348 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.825198 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16011125-3547-45a6-856c-dbc3e809e74e-tls-cert\") pod \"authorino-67dccf5fdf-p88d5\" (UID: \"16011125-3547-45a6-856c-dbc3e809e74e\") " pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.825348 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.825271 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/16011125-3547-45a6-856c-dbc3e809e74e-oidc-ca\") pod \"authorino-67dccf5fdf-p88d5\" (UID: \"16011125-3547-45a6-856c-dbc3e809e74e\") " pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.925827 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.925741 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16011125-3547-45a6-856c-dbc3e809e74e-tls-cert\") pod \"authorino-67dccf5fdf-p88d5\" (UID: \"16011125-3547-45a6-856c-dbc3e809e74e\") " pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.925827 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.925792 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/16011125-3547-45a6-856c-dbc3e809e74e-oidc-ca\") pod \"authorino-67dccf5fdf-p88d5\" (UID: \"16011125-3547-45a6-856c-dbc3e809e74e\") " pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.925990 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.925840 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jtdq4\" (UniqueName: \"kubernetes.io/projected/16011125-3547-45a6-856c-dbc3e809e74e-kube-api-access-jtdq4\") pod \"authorino-67dccf5fdf-p88d5\" (UID: \"16011125-3547-45a6-856c-dbc3e809e74e\") " pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.926493 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.926462 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/16011125-3547-45a6-856c-dbc3e809e74e-oidc-ca\") pod \"authorino-67dccf5fdf-p88d5\" (UID: \"16011125-3547-45a6-856c-dbc3e809e74e\") " pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.928119 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.928091 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16011125-3547-45a6-856c-dbc3e809e74e-tls-cert\") pod \"authorino-67dccf5fdf-p88d5\" (UID: \"16011125-3547-45a6-856c-dbc3e809e74e\") " pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.932587 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.932563 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtdq4\" (UniqueName: \"kubernetes.io/projected/16011125-3547-45a6-856c-dbc3e809e74e-kube-api-access-jtdq4\") pod \"authorino-67dccf5fdf-p88d5\" (UID: \"16011125-3547-45a6-856c-dbc3e809e74e\") " pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:16.989467 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:16.989441 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67dccf5fdf-p88d5" Jun 01 20:48:17.111778 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:17.111754 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67dccf5fdf-p88d5"] Jun 01 20:48:17.114223 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:48:17.114193 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16011125_3547_45a6_856c_dbc3e809e74e.slice/crio-bab84f6f56ae44fb91bcd5c9604aa28befa6424425d26cf796b73d5561ff581b WatchSource:0}: Error finding container bab84f6f56ae44fb91bcd5c9604aa28befa6424425d26cf796b73d5561ff581b: Status 404 returned error can't find the container with id bab84f6f56ae44fb91bcd5c9604aa28befa6424425d26cf796b73d5561ff581b Jun 01 20:48:17.610344 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:17.610309 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67dccf5fdf-p88d5" event={"ID":"16011125-3547-45a6-856c-dbc3e809e74e","Type":"ContainerStarted","Data":"cde1f9211045215070311acd4b8c627df35d6e916a41bff65bf4d2363dfc20e5"} Jun 01 20:48:17.610344 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:17.610347 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67dccf5fdf-p88d5" event={"ID":"16011125-3547-45a6-856c-dbc3e809e74e","Type":"ContainerStarted","Data":"bab84f6f56ae44fb91bcd5c9604aa28befa6424425d26cf796b73d5561ff581b"} Jun 01 20:48:17.624513 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:17.624456 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-67dccf5fdf-p88d5" podStartSLOduration=1.292410355 podStartE2EDuration="1.624416543s" podCreationTimestamp="2026-06-01 20:48:16 +0000 UTC" firstStartedPulling="2026-06-01 20:48:17.115387395 +0000 UTC m=+857.822759909" lastFinishedPulling="2026-06-01 20:48:17.447393577 +0000 UTC m=+858.154766097" observedRunningTime="2026-06-01 20:48:17.623441547 +0000 UTC m=+858.330814075" watchObservedRunningTime="2026-06-01 20:48:17.624416543 +0000 UTC m=+858.331789083" Jun 01 20:48:17.649980 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:17.649942 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cffdb4dbc-5d9cc"] Jun 01 20:48:17.650932 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:17.650762 2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" podUID="8379d005-e9b3-4523-b245-7828cc34edf6" containerName="authorino" containerID="cri-o://091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147" gracePeriod=30 Jun 01 20:48:17.910751 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:17.910722 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:48:18.036572 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.036485 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fvnx\" (UniqueName: \"kubernetes.io/projected/8379d005-e9b3-4523-b245-7828cc34edf6-kube-api-access-4fvnx\") pod \"8379d005-e9b3-4523-b245-7828cc34edf6\" (UID: \"8379d005-e9b3-4523-b245-7828cc34edf6\") " Jun 01 20:48:18.036745 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.036650 2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8379d005-e9b3-4523-b245-7828cc34edf6-tls-cert\") pod \"8379d005-e9b3-4523-b245-7828cc34edf6\" (UID: \"8379d005-e9b3-4523-b245-7828cc34edf6\") " Jun 01 20:48:18.038681 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.038649 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8379d005-e9b3-4523-b245-7828cc34edf6-kube-api-access-4fvnx" (OuterVolumeSpecName: "kube-api-access-4fvnx") pod "8379d005-e9b3-4523-b245-7828cc34edf6" (UID: "8379d005-e9b3-4523-b245-7828cc34edf6"). InnerVolumeSpecName "kube-api-access-4fvnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 20:48:18.046669 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.046637 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8379d005-e9b3-4523-b245-7828cc34edf6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "8379d005-e9b3-4523-b245-7828cc34edf6" (UID: "8379d005-e9b3-4523-b245-7828cc34edf6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 01 20:48:18.137836 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.137797 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8379d005-e9b3-4523-b245-7828cc34edf6-tls-cert\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:48:18.137836 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.137831 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fvnx\" (UniqueName: \"kubernetes.io/projected/8379d005-e9b3-4523-b245-7828cc34edf6-kube-api-access-4fvnx\") on node \"ip-10-0-141-183.ec2.internal\" DevicePath \"\"" Jun 01 20:48:18.614603 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.614562 2559 generic.go:358] "Generic (PLEG): container finished" podID="8379d005-e9b3-4523-b245-7828cc34edf6" containerID="091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147" exitCode=0 Jun 01 20:48:18.614819 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.614618 2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" Jun 01 20:48:18.614819 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.614648 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" event={"ID":"8379d005-e9b3-4523-b245-7828cc34edf6","Type":"ContainerDied","Data":"091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147"} Jun 01 20:48:18.614819 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.614683 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cffdb4dbc-5d9cc" event={"ID":"8379d005-e9b3-4523-b245-7828cc34edf6","Type":"ContainerDied","Data":"ff9cbb6d798855e7ec049dc50f2cc1ebf52e7530d5ba1a27fb7afd9664fe723c"} Jun 01 20:48:18.614819 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.614698 2559 scope.go:117] "RemoveContainer" containerID="091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147" Jun 01 20:48:18.623612 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.623595 2559 scope.go:117] "RemoveContainer" containerID="091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147" Jun 01 20:48:18.623841 ip-10-0-141-183 kubenswrapper[2559]: E0601 20:48:18.623823 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147\": container with ID starting with 091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147 not found: ID does not exist" containerID="091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147" Jun 01 20:48:18.623894 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.623849 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147"} err="failed to get container status \"091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147\": rpc error: code = NotFound desc = could not find container \"091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147\": container with ID starting with 091229a0707a25d66318339fafc1b1c945cf6e6d4456356a804c1cc6ec4ce147 not found: ID does not exist" Jun 01 20:48:18.633703 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.633678 2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cffdb4dbc-5d9cc"] Jun 01 20:48:18.640777 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:18.640745 2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5cffdb4dbc-5d9cc"] Jun 01 20:48:19.832325 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:19.832291 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8379d005-e9b3-4523-b245-7828cc34edf6" path="/var/lib/kubelet/pods/8379d005-e9b3-4523-b245-7828cc34edf6/volumes" Jun 01 20:48:20.896929 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.896896 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb"] Jun 01 20:48:20.897391 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.897370 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8379d005-e9b3-4523-b245-7828cc34edf6" containerName="authorino" Jun 01 20:48:20.897493 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.897396 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8379d005-e9b3-4523-b245-7828cc34edf6" containerName="authorino" Jun 01 20:48:20.897554 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.897493 2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="8379d005-e9b3-4523-b245-7828cc34edf6" containerName="authorino" Jun 01 20:48:20.900352 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.900330 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:20.902242 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.902221 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 01 20:48:20.902434 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.902400 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ncql6\"" Jun 01 20:48:20.902522 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.902401 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 01 20:48:20.902843 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.902826 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 01 20:48:20.914528 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:20.914508 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb"] Jun 01 20:48:21.062063 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.062031 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.062232 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.062087 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.062232 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.062159 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.062346 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.062228 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.062346 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.062271 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.062346 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.062303 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4cz9\" (UniqueName: \"kubernetes.io/projected/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-kube-api-access-c4cz9\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.163180 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.163097 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.163180 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.163150 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.163180 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.163176 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.163407 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.163202 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.163407 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.163320 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4cz9\" (UniqueName: \"kubernetes.io/projected/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-kube-api-access-c4cz9\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.163407 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.163385 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.163594 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.163558 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.163708 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.163685 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.163787 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.163754 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.165946 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.165915 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.166305 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.166284 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.170858 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.170827 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4cz9\" (UniqueName: \"kubernetes.io/projected/b40e33c7-2413-4faf-8a3f-b1bf20ba687c-kube-api-access-c4cz9\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb\" (UID: \"b40e33c7-2413-4faf-8a3f-b1bf20ba687c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.211450 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.211407 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:21.339523 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.339347 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb"] Jun 01 20:48:21.341886 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:48:21.341860 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40e33c7_2413_4faf_8a3f_b1bf20ba687c.slice/crio-1ee71e41f7a66cb3b05f252792385f306d33b19967484e3c805ce819824d2d13 WatchSource:0}: Error finding container 1ee71e41f7a66cb3b05f252792385f306d33b19967484e3c805ce819824d2d13: Status 404 returned error can't find the container with id 1ee71e41f7a66cb3b05f252792385f306d33b19967484e3c805ce819824d2d13 Jun 01 20:48:21.627197 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:21.627104 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" event={"ID":"b40e33c7-2413-4faf-8a3f-b1bf20ba687c","Type":"ContainerStarted","Data":"1ee71e41f7a66cb3b05f252792385f306d33b19967484e3c805ce819824d2d13"} Jun 01 20:48:30.656201 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:30.656164 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" event={"ID":"b40e33c7-2413-4faf-8a3f-b1bf20ba687c","Type":"ContainerStarted","Data":"6325d87a8ed149d9699522be6ab6fcbe69d8bd87603cc9ec4e62e4fa0cdaec37"} Jun 01 20:48:31.660468 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:31.660400 2559 generic.go:358] "Generic (PLEG): container finished" podID="b40e33c7-2413-4faf-8a3f-b1bf20ba687c" containerID="6325d87a8ed149d9699522be6ab6fcbe69d8bd87603cc9ec4e62e4fa0cdaec37" exitCode=0 Jun 01 20:48:31.660885 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:31.660485 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" event={"ID":"b40e33c7-2413-4faf-8a3f-b1bf20ba687c","Type":"ContainerDied","Data":"6325d87a8ed149d9699522be6ab6fcbe69d8bd87603cc9ec4e62e4fa0cdaec37"} Jun 01 20:48:33.667960 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:33.667924 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" event={"ID":"b40e33c7-2413-4faf-8a3f-b1bf20ba687c","Type":"ContainerStarted","Data":"8ac3bce6a939506f3f936cec148ea1f97955ac0a8f5c95ef7d25175718020c30"} Jun 01 20:48:33.684991 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:33.684946 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" podStartSLOduration=2.221785209 podStartE2EDuration="13.684932028s" podCreationTimestamp="2026-06-01 20:48:20 +0000 UTC" firstStartedPulling="2026-06-01 20:48:21.344085133 +0000 UTC m=+862.051457648" lastFinishedPulling="2026-06-01 20:48:32.807231949 +0000 UTC m=+873.514604467" observedRunningTime="2026-06-01 20:48:33.683341677 +0000 UTC m=+874.390714214" watchObservedRunningTime="2026-06-01 20:48:33.684932028 +0000 UTC m=+874.392304565" Jun 01 20:48:36.593338 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.593300 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85"] Jun 01 20:48:36.600070 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.600041 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.602184 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.602161 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 01 20:48:36.606448 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.606403 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85"] Jun 01 20:48:36.698566 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.698526 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.698742 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.698632 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.698742 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.698662 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.698742 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.698683 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f618cc21-a084-4eab-b5af-8569e5f1d85f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.698742 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.698699 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.698742 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.698718 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h68hd\" (UniqueName: \"kubernetes.io/projected/f618cc21-a084-4eab-b5af-8569e5f1d85f-kube-api-access-h68hd\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.799821 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.799778 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.799999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.799857 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.799999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.799884 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.799999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.799900 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f618cc21-a084-4eab-b5af-8569e5f1d85f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.799999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.799918 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.799999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.799937 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h68hd\" (UniqueName: \"kubernetes.io/projected/f618cc21-a084-4eab-b5af-8569e5f1d85f-kube-api-access-h68hd\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.800241 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.800218 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.800301 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.800279 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.800345 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.800300 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.802190 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.802158 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f618cc21-a084-4eab-b5af-8569e5f1d85f-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.802401 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.802383 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f618cc21-a084-4eab-b5af-8569e5f1d85f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.806826 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.806804 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h68hd\" (UniqueName: \"kubernetes.io/projected/f618cc21-a084-4eab-b5af-8569e5f1d85f-kube-api-access-h68hd\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85\" (UID: \"f618cc21-a084-4eab-b5af-8569e5f1d85f\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:36.911979 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:36.911944 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:37.041840 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:37.041803 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85"] Jun 01 20:48:37.044723 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:48:37.044696 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf618cc21_a084_4eab_b5af_8569e5f1d85f.slice/crio-cb5114a79466cda27a09434e9f3433485f3d5ec3d359253b8391cb1232844b96 WatchSource:0}: Error finding container cb5114a79466cda27a09434e9f3433485f3d5ec3d359253b8391cb1232844b96: Status 404 returned error can't find the container with id cb5114a79466cda27a09434e9f3433485f3d5ec3d359253b8391cb1232844b96 Jun 01 20:48:37.682590 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:37.682552 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" event={"ID":"f618cc21-a084-4eab-b5af-8569e5f1d85f","Type":"ContainerStarted","Data":"40c8ea1dc96bf0603bec224881f329d06be82cb37f97e1aff19779ec451d97c7"} Jun 01 20:48:37.682590 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:37.682593 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" event={"ID":"f618cc21-a084-4eab-b5af-8569e5f1d85f","Type":"ContainerStarted","Data":"cb5114a79466cda27a09434e9f3433485f3d5ec3d359253b8391cb1232844b96"} Jun 01 20:48:38.686737 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:38.686644 2559 generic.go:358] "Generic (PLEG): container finished" podID="f618cc21-a084-4eab-b5af-8569e5f1d85f" containerID="40c8ea1dc96bf0603bec224881f329d06be82cb37f97e1aff19779ec451d97c7" exitCode=0 Jun 01 20:48:38.687100 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:38.686734 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" event={"ID":"f618cc21-a084-4eab-b5af-8569e5f1d85f","Type":"ContainerDied","Data":"40c8ea1dc96bf0603bec224881f329d06be82cb37f97e1aff19779ec451d97c7"} Jun 01 20:48:39.691769 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:39.691737 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" event={"ID":"f618cc21-a084-4eab-b5af-8569e5f1d85f","Type":"ContainerStarted","Data":"4a56382eaac8f09a3f825c87daf9ba3c9dec64f361bd50147970780f37c97957"} Jun 01 20:48:39.708432 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:39.708375 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" podStartSLOduration=3.467807521 podStartE2EDuration="3.708358254s" podCreationTimestamp="2026-06-01 20:48:36 +0000 UTC" firstStartedPulling="2026-06-01 20:48:38.687209101 +0000 UTC m=+879.394581616" lastFinishedPulling="2026-06-01 20:48:38.927759831 +0000 UTC m=+879.635132349" observedRunningTime="2026-06-01 20:48:39.708063078 +0000 UTC m=+880.415435624" watchObservedRunningTime="2026-06-01 20:48:39.708358254 +0000 UTC m=+880.415730792" Jun 01 20:48:41.211851 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:41.211813 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:41.212316 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:41.211866 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:41.224271 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:41.224245 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:41.710208 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:41.710180 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb" Jun 01 20:48:41.995521 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:41.995441 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6"] Jun 01 20:48:41.999032 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:41.999011 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.001017 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.000990 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 01 20:48:42.008660 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.008637 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6"] Jun 01 20:48:42.044362 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.044337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.044498 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.044373 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58499ddc-f4a8-44ee-ab00-8e0f439afc88-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.044498 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.044392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx52q\" (UniqueName: \"kubernetes.io/projected/58499ddc-f4a8-44ee-ab00-8e0f439afc88-kube-api-access-sx52q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.044609 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.044531 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.044609 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.044602 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.044714 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.044624 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.145637 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.145598 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.145805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.145647 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58499ddc-f4a8-44ee-ab00-8e0f439afc88-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.145805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.145665 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sx52q\" (UniqueName: \"kubernetes.io/projected/58499ddc-f4a8-44ee-ab00-8e0f439afc88-kube-api-access-sx52q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.145805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.145717 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.145805 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.145771 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.145968 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.145805 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.146051 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.146026 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.146106 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.146086 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.146238 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.146218 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.148061 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.148035 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58499ddc-f4a8-44ee-ab00-8e0f439afc88-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.148182 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.148118 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58499ddc-f4a8-44ee-ab00-8e0f439afc88-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.153172 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.153152 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx52q\" (UniqueName: \"kubernetes.io/projected/58499ddc-f4a8-44ee-ab00-8e0f439afc88-kube-api-access-sx52q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6\" (UID: \"58499ddc-f4a8-44ee-ab00-8e0f439afc88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.329552 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.329470 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:42.454957 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.454934 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6"] Jun 01 20:48:42.456842 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:48:42.456812 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58499ddc_f4a8_44ee_ab00_8e0f439afc88.slice/crio-78c8aeb67a35923c62cd498a79056cb54cd9ed008be1002356d6dbcd74780872 WatchSource:0}: Error finding container 78c8aeb67a35923c62cd498a79056cb54cd9ed008be1002356d6dbcd74780872: Status 404 returned error can't find the container with id 78c8aeb67a35923c62cd498a79056cb54cd9ed008be1002356d6dbcd74780872 Jun 01 20:48:42.703289 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.703250 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" event={"ID":"58499ddc-f4a8-44ee-ab00-8e0f439afc88","Type":"ContainerStarted","Data":"ecfe0ec8fc937fb18e4bfa6256de97f4b1e1db168b1f38e4572336b05d241ef8"} Jun 01 20:48:42.703289 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:42.703289 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" event={"ID":"58499ddc-f4a8-44ee-ab00-8e0f439afc88","Type":"ContainerStarted","Data":"78c8aeb67a35923c62cd498a79056cb54cd9ed008be1002356d6dbcd74780872"} Jun 01 20:48:44.712500 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:44.712466 2559 generic.go:358] "Generic (PLEG): container finished" podID="58499ddc-f4a8-44ee-ab00-8e0f439afc88" containerID="ecfe0ec8fc937fb18e4bfa6256de97f4b1e1db168b1f38e4572336b05d241ef8" exitCode=0 Jun 01 20:48:44.712924 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:44.712505 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" event={"ID":"58499ddc-f4a8-44ee-ab00-8e0f439afc88","Type":"ContainerDied","Data":"ecfe0ec8fc937fb18e4bfa6256de97f4b1e1db168b1f38e4572336b05d241ef8"} Jun 01 20:48:45.717606 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:45.717569 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" event={"ID":"58499ddc-f4a8-44ee-ab00-8e0f439afc88","Type":"ContainerStarted","Data":"7b83d7b0e2734e47e266ad7102e9f651522d65523c8fb44da11466771af52fa4"} Jun 01 20:48:45.733582 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:45.733528 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" podStartSLOduration=4.524867411 podStartE2EDuration="4.7335119s" podCreationTimestamp="2026-06-01 20:48:41 +0000 UTC" firstStartedPulling="2026-06-01 20:48:44.713102192 +0000 UTC m=+885.420474707" lastFinishedPulling="2026-06-01 20:48:44.921746471 +0000 UTC m=+885.629119196" observedRunningTime="2026-06-01 20:48:45.73236331 +0000 UTC m=+886.439735847" watchObservedRunningTime="2026-06-01 20:48:45.7335119 +0000 UTC m=+886.440884437" Jun 01 20:48:46.912735 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:46.912680 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:46.913163 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:46.912760 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:46.925099 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:46.925077 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:47.740193 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:47.736402 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85" Jun 01 20:48:52.330182 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:52.330143 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:52.330182 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:52.330190 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:52.343076 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:52.343045 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:52.754924 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:52.754887 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6" Jun 01 20:48:59.797442 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.797400 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2"] Jun 01 20:48:59.814604 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.814579 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:48:59.816927 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.816900 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2"] Jun 01 20:48:59.818050 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.818026 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 01 20:48:59.909884 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.909843 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:48:59.910039 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.909899 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3e7143-fbde-49cb-86b4-77a62a2c5488-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:48:59.910039 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.909977 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2dvp\" (UniqueName: \"kubernetes.io/projected/7d3e7143-fbde-49cb-86b4-77a62a2c5488-kube-api-access-n2dvp\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:48:59.910126 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.910094 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:48:59.910126 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.910113 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:48:59.910202 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:48:59.910142 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.010859 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.010784 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n2dvp\" (UniqueName: \"kubernetes.io/projected/7d3e7143-fbde-49cb-86b4-77a62a2c5488-kube-api-access-n2dvp\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.011052 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.010939 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.011052 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.010966 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.011052 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.010994 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.011052 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.011025 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.011255 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.011073 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3e7143-fbde-49cb-86b4-77a62a2c5488-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.011341 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.011319 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.011404 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.011378 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.011505 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.011408 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.013311 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.013274 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d3e7143-fbde-49cb-86b4-77a62a2c5488-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.013568 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.013553 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d3e7143-fbde-49cb-86b4-77a62a2c5488-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.018761 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.018710 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2dvp\" (UniqueName: \"kubernetes.io/projected/7d3e7143-fbde-49cb-86b4-77a62a2c5488-kube-api-access-n2dvp\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2\" (UID: \"7d3e7143-fbde-49cb-86b4-77a62a2c5488\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.127560 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.127527 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:00.260889 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.260695 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2"] Jun 01 20:49:00.263138 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:49:00.263108 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d3e7143_fbde_49cb_86b4_77a62a2c5488.slice/crio-8c7634f32b27ac9c6b57906f3b0ae12289c4f87c03ad2ea4879fae1baba4cbf2 WatchSource:0}: Error finding container 8c7634f32b27ac9c6b57906f3b0ae12289c4f87c03ad2ea4879fae1baba4cbf2: Status 404 returned error can't find the container with id 8c7634f32b27ac9c6b57906f3b0ae12289c4f87c03ad2ea4879fae1baba4cbf2 Jun 01 20:49:00.772307 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.772273 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" event={"ID":"7d3e7143-fbde-49cb-86b4-77a62a2c5488","Type":"ContainerStarted","Data":"54151fceed9251f4d91086c2919b6b320a768cef3407ee17db0e6c32408e5df7"} Jun 01 20:49:00.772307 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:00.772309 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" event={"ID":"7d3e7143-fbde-49cb-86b4-77a62a2c5488","Type":"ContainerStarted","Data":"8c7634f32b27ac9c6b57906f3b0ae12289c4f87c03ad2ea4879fae1baba4cbf2"} Jun 01 20:49:01.778191 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:01.778156 2559 generic.go:358] "Generic (PLEG): container finished" podID="7d3e7143-fbde-49cb-86b4-77a62a2c5488" containerID="54151fceed9251f4d91086c2919b6b320a768cef3407ee17db0e6c32408e5df7" exitCode=0 Jun 01 20:49:01.778596 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:01.778243 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" event={"ID":"7d3e7143-fbde-49cb-86b4-77a62a2c5488","Type":"ContainerDied","Data":"54151fceed9251f4d91086c2919b6b320a768cef3407ee17db0e6c32408e5df7"} Jun 01 20:49:02.784081 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:02.784040 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" event={"ID":"7d3e7143-fbde-49cb-86b4-77a62a2c5488","Type":"ContainerStarted","Data":"7131dd8ef114a689a93f73e217021bfab6c2dfbba48317be349e1c927a0aea2f"} Jun 01 20:49:02.802130 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:02.802073 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" podStartSLOduration=3.538738031 podStartE2EDuration="3.80205896s" podCreationTimestamp="2026-06-01 20:48:59 +0000 UTC" firstStartedPulling="2026-06-01 20:49:01.778751267 +0000 UTC m=+902.486123782" lastFinishedPulling="2026-06-01 20:49:02.042072183 +0000 UTC m=+902.749444711" observedRunningTime="2026-06-01 20:49:02.800019063 +0000 UTC m=+903.507391599" watchObservedRunningTime="2026-06-01 20:49:02.80205896 +0000 UTC m=+903.509431662" Jun 01 20:49:10.128596 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:10.128554 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:10.129025 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:10.128652 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:10.140929 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:10.140902 2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:49:10.822028 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:49:10.821998 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2" Jun 01 20:54:53.078999 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:54:53.078925 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67dccf5fdf-p88d5_16011125-3547-45a6-856c-dbc3e809e74e/authorino/0.log" Jun 01 20:54:57.988643 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:54:57.988616 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-75cfcc4598-mvlsw_999de3e3-d690-4ca4-a18c-18be3495bd18/maas-api/0.log" Jun 01 20:54:58.103768 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:54:58.103731 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6745f5f64f-tpdqh_5858cfb8-74b6-4f11-817d-90e17f3d385c/manager/0.log" Jun 01 20:54:58.444011 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:54:58.443982 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64857648b-8g9x5_28f3dd61-4614-4e31-b334-974983fa1b2c/copy-manifests/0.log" Jun 01 20:54:58.450563 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:54:58.450534 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64857648b-8g9x5_28f3dd61-4614-4e31-b334-974983fa1b2c/manager/0.log" Jun 01 20:55:00.079482 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:00.079454 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67dccf5fdf-p88d5_16011125-3547-45a6-856c-dbc3e809e74e/authorino/0.log" Jun 01 20:55:00.432416 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:00.432388 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-nc62q_f239ad67-4e5d-47b0-a6dc-682e9d5eba23/kuadrant-console-plugin/0.log" Jun 01 20:55:01.469656 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:01.469627 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b9fc46-4jczs_0807bb2a-960e-4504-bcba-10190766f686/kube-auth-proxy/0.log" Jun 01 20:55:02.248732 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:02.248699 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb_b40e33c7-2413-4faf-8a3f-b1bf20ba687c/storage-initializer/0.log" Jun 01 20:55:02.255526 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:02.255499 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-8lccb_b40e33c7-2413-4faf-8a3f-b1bf20ba687c/main/0.log" Jun 01 20:55:02.481653 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:02.481617 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85_f618cc21-a084-4eab-b5af-8569e5f1d85f/storage-initializer/0.log" Jun 01 20:55:02.488454 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:02.488415 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-9ln85_f618cc21-a084-4eab-b5af-8569e5f1d85f/main/0.log" Jun 01 20:55:02.596904 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:02.596825 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6_58499ddc-f4a8-44ee-ab00-8e0f439afc88/storage-initializer/0.log" Jun 01 20:55:02.603120 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:02.603097 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9fp4v6_58499ddc-f4a8-44ee-ab00-8e0f439afc88/main/0.log" Jun 01 20:55:02.714119 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:02.714090 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2_7d3e7143-fbde-49cb-86b4-77a62a2c5488/storage-initializer/0.log" Jun 01 20:55:02.721119 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:02.721095 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-j8jq2_7d3e7143-fbde-49cb-86b4-77a62a2c5488/main/0.log" Jun 01 20:55:09.997984 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:09.997957 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p8f5k_752796d9-4fc8-4dd8-b91c-611bdc542833/global-pull-secret-syncer/0.log" Jun 01 20:55:10.039027 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:10.038998 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9z8pf_13529312-4c7d-4dfe-bebb-61522388b930/konnectivity-agent/0.log" Jun 01 20:55:10.140975 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:10.140946 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-183.ec2.internal_b297c8edefffe884478fc112206a107d/haproxy/0.log" Jun 01 20:55:14.791899 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:14.791867 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67dccf5fdf-p88d5_16011125-3547-45a6-856c-dbc3e809e74e/authorino/0.log" Jun 01 20:55:14.867471 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:14.867441 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-nc62q_f239ad67-4e5d-47b0-a6dc-682e9d5eba23/kuadrant-console-plugin/0.log" Jun 01 20:55:16.414832 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.414801 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_427be6f2-acd6-4964-b6b4-4da497d87312/alertmanager/0.log" Jun 01 20:55:16.437963 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.437935 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_427be6f2-acd6-4964-b6b4-4da497d87312/config-reloader/0.log" Jun 01 20:55:16.477811 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.477781 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_427be6f2-acd6-4964-b6b4-4da497d87312/kube-rbac-proxy-web/0.log" Jun 01 20:55:16.511417 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.511395 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_427be6f2-acd6-4964-b6b4-4da497d87312/kube-rbac-proxy/0.log" Jun 01 20:55:16.537729 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.537700 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_427be6f2-acd6-4964-b6b4-4da497d87312/kube-rbac-proxy-metric/0.log" Jun 01 20:55:16.561492 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.561466 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_427be6f2-acd6-4964-b6b4-4da497d87312/prom-label-proxy/0.log" Jun 01 20:55:16.584890 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.584870 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_427be6f2-acd6-4964-b6b4-4da497d87312/init-config-reloader/0.log" Jun 01 20:55:16.741745 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.741660 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5fcb55cc7d-2xs4z_1654d8b6-07a6-4231-92c2-3b16961540b0/metrics-server/0.log" Jun 01 20:55:16.960242 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.960218 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxgxd_5ddae794-85d0-4260-be7c-7f486ab7a23f/node-exporter/0.log" Jun 01 20:55:16.979190 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.979151 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxgxd_5ddae794-85d0-4260-be7c-7f486ab7a23f/kube-rbac-proxy/0.log" Jun 01 20:55:16.997976 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:16.997921 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxgxd_5ddae794-85d0-4260-be7c-7f486ab7a23f/init-textfile/0.log" Jun 01 20:55:17.352050 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:17.351968 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-d6cpp_188b747c-a516-4422-af9e-b878c6b2bbf2/prometheus-operator-admission-webhook/0.log" Jun 01 20:55:17.393368 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:17.393338 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d7b8b5d4b-sw9rn_3c13d5ee-d8a8-4a4c-a1f8-663979899dcc/telemeter-client/0.log" Jun 01 20:55:17.416226 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:17.416201 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d7b8b5d4b-sw9rn_3c13d5ee-d8a8-4a4c-a1f8-663979899dcc/reload/0.log" Jun 01 20:55:17.434381 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:17.434359 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d7b8b5d4b-sw9rn_3c13d5ee-d8a8-4a4c-a1f8-663979899dcc/kube-rbac-proxy/0.log" Jun 01 20:55:20.029894 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.029859 2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk"] Jun 01 20:55:20.033249 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.033225 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.035306 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.035281 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gxqp2\"/\"openshift-service-ca.crt\"" Jun 01 20:55:20.035712 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.035696 2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gxqp2\"/\"default-dockercfg-77s6m\"" Jun 01 20:55:20.035784 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.035697 2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gxqp2\"/\"kube-root-ca.crt\"" Jun 01 20:55:20.039848 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.039825 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk"] Jun 01 20:55:20.180208 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.180173 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-proc\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.180395 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.180224 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-sys\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.180395 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.180286 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-lib-modules\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.180395 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.180311 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gfl9\" (UniqueName: \"kubernetes.io/projected/81a419a5-67ed-4f91-9103-2be64d50a111-kube-api-access-4gfl9\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.180395 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.180360 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-podres\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.281793 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.281696 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-lib-modules\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.281793 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.281735 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4gfl9\" (UniqueName: \"kubernetes.io/projected/81a419a5-67ed-4f91-9103-2be64d50a111-kube-api-access-4gfl9\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.282033 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.281802 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-podres\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.282033 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.281847 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-proc\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.282033 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.281887 2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-sys\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.282033 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.281898 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-lib-modules\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.282033 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.281948 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-podres\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.282033 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.281979 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-proc\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.282033 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.282009 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81a419a5-67ed-4f91-9103-2be64d50a111-sys\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.289023 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.289001 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gfl9\" (UniqueName: \"kubernetes.io/projected/81a419a5-67ed-4f91-9103-2be64d50a111-kube-api-access-4gfl9\") pod \"perf-node-gather-daemonset-zg7dk\" (UID: \"81a419a5-67ed-4f91-9103-2be64d50a111\") " pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.343792 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.343756 2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:20.473845 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.473732 2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk"] Jun 01 20:55:20.476952 ip-10-0-141-183 kubenswrapper[2559]: W0601 20:55:20.476921 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod81a419a5_67ed_4f91_9103_2be64d50a111.slice/crio-a3034b3a7f67e505007a0cf478f0d9fabfa3fb0a721afedf121b86264d9fb2d9 WatchSource:0}: Error finding container a3034b3a7f67e505007a0cf478f0d9fabfa3fb0a721afedf121b86264d9fb2d9: Status 404 returned error can't find the container with id a3034b3a7f67e505007a0cf478f0d9fabfa3fb0a721afedf121b86264d9fb2d9 Jun 01 20:55:20.478769 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:20.478744 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 01 20:55:21.050641 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:21.050599 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" event={"ID":"81a419a5-67ed-4f91-9103-2be64d50a111","Type":"ContainerStarted","Data":"7e5a3a582c0bceb4c9b607058a65dea1b53b1543190dfe984f8ca9a556a5d8eb"} Jun 01 20:55:21.050641 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:21.050642 2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" event={"ID":"81a419a5-67ed-4f91-9103-2be64d50a111","Type":"ContainerStarted","Data":"a3034b3a7f67e505007a0cf478f0d9fabfa3fb0a721afedf121b86264d9fb2d9"} Jun 01 20:55:21.051072 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:21.050667 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:21.067037 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:21.066989 2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" podStartSLOduration=1.066975752 podStartE2EDuration="1.066975752s" podCreationTimestamp="2026-06-01 20:55:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-01 20:55:21.064983248 +0000 UTC m=+1281.772355799" watchObservedRunningTime="2026-06-01 20:55:21.066975752 +0000 UTC m=+1281.774348286" Jun 01 20:55:21.130476 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:21.130448 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qf2m5_f014dc94-fbcf-4028-b18b-b7ecab010b40/dns/0.log" Jun 01 20:55:21.148331 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:21.148312 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qf2m5_f014dc94-fbcf-4028-b18b-b7ecab010b40/kube-rbac-proxy/0.log" Jun 01 20:55:21.294273 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:21.294241 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xp92g_d5979744-4da4-40e1-9001-477372c8d2c0/dns-node-resolver/0.log" Jun 01 20:55:21.834760 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:21.834732 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cx8lv_c0d1f181-3281-404c-a019-c7a99930d33d/node-ca/0.log" Jun 01 20:55:22.757683 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:22.757648 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b9fc46-4jczs_0807bb2a-960e-4504-bcba-10190766f686/kube-auth-proxy/0.log" Jun 01 20:55:23.386729 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:23.386698 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v8zqc_24c4bd1b-12a5-4018-b753-83ec3ea7996c/serve-healthcheck-canary/0.log" Jun 01 20:55:24.000367 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:24.000337 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t9ffl_93623e06-a28e-455a-aa34-3c7e446e8219/kube-rbac-proxy/0.log" Jun 01 20:55:24.018764 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:24.018738 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t9ffl_93623e06-a28e-455a-aa34-3c7e446e8219/exporter/0.log" Jun 01 20:55:24.038133 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:24.038109 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t9ffl_93623e06-a28e-455a-aa34-3c7e446e8219/extractor/0.log" Jun 01 20:55:26.097217 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:26.097187 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-75cfcc4598-mvlsw_999de3e3-d690-4ca4-a18c-18be3495bd18/maas-api/0.log" Jun 01 20:55:26.119849 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:26.119823 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6745f5f64f-tpdqh_5858cfb8-74b6-4f11-817d-90e17f3d385c/manager/0.log" Jun 01 20:55:26.220235 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:26.220207 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64857648b-8g9x5_28f3dd61-4614-4e31-b334-974983fa1b2c/manager/0.log" Jun 01 20:55:26.237157 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:26.237138 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64857648b-8g9x5_28f3dd61-4614-4e31-b334-974983fa1b2c/copy-manifests/0.log" Jun 01 20:55:27.063060 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:27.063033 2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gxqp2/perf-node-gather-daemonset-zg7dk" Jun 01 20:55:27.706085 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:27.706055 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58fff47c47-4r49s_d18633f5-00a1-460b-bd4f-be239aa3025f/manager/0.log" Jun 01 20:55:27.762180 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:27.762157 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-9l7ns_9e982334-2307-4c5b-b722-0233e8ee4a78/openshift-lws-operator/0.log" Jun 01 20:55:34.228591 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.228556 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd28x_9601c936-7a8f-401f-80b3-0e4a53bd248a/kube-multus-additional-cni-plugins/0.log" Jun 01 20:55:34.248511 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.248486 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd28x_9601c936-7a8f-401f-80b3-0e4a53bd248a/egress-router-binary-copy/0.log" Jun 01 20:55:34.267612 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.267592 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd28x_9601c936-7a8f-401f-80b3-0e4a53bd248a/cni-plugins/0.log" Jun 01 20:55:34.286511 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.286490 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd28x_9601c936-7a8f-401f-80b3-0e4a53bd248a/bond-cni-plugin/0.log" Jun 01 20:55:34.304154 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.304135 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd28x_9601c936-7a8f-401f-80b3-0e4a53bd248a/routeoverride-cni/0.log" Jun 01 20:55:34.321383 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.321365 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd28x_9601c936-7a8f-401f-80b3-0e4a53bd248a/whereabouts-cni-bincopy/0.log" Jun 01 20:55:34.341801 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.341778 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pd28x_9601c936-7a8f-401f-80b3-0e4a53bd248a/whereabouts-cni/0.log" Jun 01 20:55:34.404804 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.404774 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q95l8_ab4d0b21-4002-4ec4-8bc4-5e4a7e3225d9/kube-multus/0.log" Jun 01 20:55:34.503408 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.503341 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z6xn6_818c75c1-e3d8-4a90-89de-ade8164c6d4a/network-metrics-daemon/0.log" Jun 01 20:55:34.520516 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:34.520496 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z6xn6_818c75c1-e3d8-4a90-89de-ade8164c6d4a/kube-rbac-proxy/0.log" Jun 01 20:55:35.995772 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:35.995743 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knvtx_86c772bf-658d-41c4-81e4-f4642d3e47d6/ovn-controller/0.log" Jun 01 20:55:36.023219 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:36.023196 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knvtx_86c772bf-658d-41c4-81e4-f4642d3e47d6/ovn-acl-logging/0.log" Jun 01 20:55:36.043976 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:36.043945 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knvtx_86c772bf-658d-41c4-81e4-f4642d3e47d6/kube-rbac-proxy-node/0.log" Jun 01 20:55:36.063208 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:36.063183 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knvtx_86c772bf-658d-41c4-81e4-f4642d3e47d6/kube-rbac-proxy-ovn-metrics/0.log" Jun 01 20:55:36.080037 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:36.080014 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knvtx_86c772bf-658d-41c4-81e4-f4642d3e47d6/northd/0.log" Jun 01 20:55:36.098182 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:36.098162 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knvtx_86c772bf-658d-41c4-81e4-f4642d3e47d6/nbdb/0.log" Jun 01 20:55:36.115758 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:36.115741 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knvtx_86c772bf-658d-41c4-81e4-f4642d3e47d6/sbdb/0.log" Jun 01 20:55:36.200789 ip-10-0-141-183 kubenswrapper[2559]: I0601 20:55:36.200768 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-knvtx_86c772bf-658d-41c4-81e4-f4642d3e47d6/ovnkube-controller/0.log"