Jun 10 12:36:49.678385 ip-10-0-138-99 systemd[1]: Starting Kubernetes Kubelet... Jun 10 12:36:50.082394 ip-10-0-138-99 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 12:36:50.082394 ip-10-0-138-99 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 10 12:36:50.082394 ip-10-0-138-99 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 12:36:50.082394 ip-10-0-138-99 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 10 12:36:50.082394 ip-10-0-138-99 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 12:36:50.084977 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.084881 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 10 12:36:50.090256 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090234 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:36:50.090256 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090255 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:36:50.090256 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090259 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090264 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090268 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090272 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090274 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090277 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090281 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090283 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090286 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090288 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090291 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090295 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090297 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090300 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090303 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090305 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090308 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090310 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090312 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:36:50.090401 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090315 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090318 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090320 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090323 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090326 2582 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090329 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090331 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090334 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090338 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090341 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090343 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090346 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090349 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090352 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090355 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090358 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090362 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090367 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090370 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:36:50.090858 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090373 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090375 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090378 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090380 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090383 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090386 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090388 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090391 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090393 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090396 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090398 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090401 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090404 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090407 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090410 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090413 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090416 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090418 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090420 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090423 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:36:50.091342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090425 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090427 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090430 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090432 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090435 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090437 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090439 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090442 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090445 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090448 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090450 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090453 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090455 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090458 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090460 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090463 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090466 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090468 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090470 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090473 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:36:50.091832 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090475 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090478 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090480 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090483 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090486 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090489 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090492 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090494 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090497 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090500 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090502 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090505 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090507 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090509 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090512 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090514 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090517 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090519 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.090522 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091259 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:36:50.092368 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091267 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091270 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091273 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091276 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091278 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091281 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091284 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091286 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091288 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091291 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091293 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091296 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091299 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091301 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091305 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091307 2582 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091310 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091313 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091316 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091318 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:36:50.092831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091321 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091323 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091326 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091328 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091330 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091333 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091335 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091339 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091343 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091347 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091351 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091354 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091358 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091361 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091364 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091367 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091370 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091373 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091376 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:36:50.093351 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091379 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091382 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091385 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091387 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091416 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091420 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091423 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091426 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091429 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091431 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091434 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091437 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091440 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091443 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091446 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091449 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091451 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091454 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091457 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091459 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:36:50.093811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091462 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091464 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091467 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091470 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091473 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091476 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091479 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091481 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091484 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091486 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091489 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091491 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091495 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091497 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091499 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091502 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091504 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091507 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091509 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091512 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:36:50.094323 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091514 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091517 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091519 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091523 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091525 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091527 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091530 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091533 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091536 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091538 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091541 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091543 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091546 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091548 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091552 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091555 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091558 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091561 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091564 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:36:50.094816 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091947 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091953 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091956 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091959 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091962 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091965 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091968 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091970 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091973 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091975 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091978 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091980 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091983 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091985 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091989 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091993 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091996 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.091999 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092002 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092005 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:36:50.095295 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092009 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092011 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092014 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092017 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092019 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092022 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092024 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092026 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092029 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092031 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092035 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092037 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092040 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092042 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092044 2582 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092047 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092049 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092052 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092054 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092056 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:36:50.095803 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092059 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092061 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092064 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092066 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092068 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092071 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092073 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092076 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092078 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092081 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092083 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092086 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092089 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092091 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092109 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092111 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092114 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092116 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092119 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092122 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:36:50.096302 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092124 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092127 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092129 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092132 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092134 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092137 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092140 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092143 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092145 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092147 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092150 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092153 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092155 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092158 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092160 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092163 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092165 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092167 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092170 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092172 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:36:50.096799 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092175 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092177 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092182 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092184 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092187 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092190 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092192 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092195 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092198 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092200 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092203 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092205 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092208 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092210 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092214 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092217 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092220 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092222 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092226 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092304 2582 flags.go:64] FLAG: --address="0.0.0.0" Jun 10 12:36:50.097300 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092314 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092325 2582 flags.go:64] FLAG: --anonymous-auth="true" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092331 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092338 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092342 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092348 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092354 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092357 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092360 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092363 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092366 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092369 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092372 2582 flags.go:64] FLAG: --cgroup-root="" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092375 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092378 2582 flags.go:64] FLAG: --client-ca-file="" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092382 2582 flags.go:64] FLAG: --cloud-provider="external" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092385 2582 flags.go:64] FLAG: --cluster-dns="[]" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092390 2582 flags.go:64] FLAG: --cluster-domain="" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092392 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092395 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092398 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092402 2582 flags.go:64] FLAG: --container-log-max-files="5" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092405 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 10 12:36:50.097791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092408 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092411 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092414 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092416 2582 flags.go:64] FLAG: --contention-profiling="false" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092419 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092422 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092425 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092428 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092432 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092435 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092438 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092441 2582 flags.go:64] FLAG: --enable-load-reader="false" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092444 2582 flags.go:64] FLAG: --enable-server="true" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092447 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092451 2582 flags.go:64] FLAG: --event-burst="100" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092454 2582 flags.go:64] FLAG: --event-qps="50" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092457 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092460 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092463 2582 flags.go:64] FLAG: --eviction-hard="" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092467 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092469 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092472 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092475 2582 flags.go:64] FLAG: --eviction-soft="" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092478 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092480 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 10 12:36:50.098369 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092484 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092487 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092489 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092492 2582 flags.go:64] FLAG: --fail-swap-on="true" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092499 2582 flags.go:64] FLAG: --feature-gates="" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092502 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092505 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092508 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092511 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092514 2582 flags.go:64] FLAG: --healthz-port="10248" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092517 2582 flags.go:64] FLAG: --help="false" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092520 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092523 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092525 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092528 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092531 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092534 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092537 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092540 2582 flags.go:64] FLAG: --image-service-endpoint="" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092543 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092545 2582 flags.go:64] FLAG: --kube-api-burst="100" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092548 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092553 2582 flags.go:64] FLAG: --kube-api-qps="50" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092556 2582 flags.go:64] FLAG: --kube-reserved="" Jun 10 12:36:50.099048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092558 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092561 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092564 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092566 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092569 2582 flags.go:64] FLAG: --lock-file="" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092572 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092575 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092578 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092584 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092587 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092590 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092592 2582 flags.go:64] FLAG: --logging-format="text" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092596 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092600 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092603 2582 flags.go:64] FLAG: --manifest-url="" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092606 2582 flags.go:64] FLAG: --manifest-url-header="" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092610 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092613 2582 flags.go:64] FLAG: --max-open-files="1000000" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092618 2582 flags.go:64] FLAG: --max-pods="110" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092620 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092624 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092626 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092629 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092632 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092634 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 10 12:36:50.099646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092637 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092644 2582 flags.go:64] FLAG: --node-status-max-images="50" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092647 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092650 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092653 2582 flags.go:64] FLAG: --pod-cidr="" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092657 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092663 2582 flags.go:64] FLAG: --pod-manifest-path="" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092666 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092669 2582 flags.go:64] FLAG: --pods-per-core="0" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092672 2582 flags.go:64] FLAG: --port="10250" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092675 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092678 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05d590b93f28910f2" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092681 2582 flags.go:64] FLAG: --qos-reserved="" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092684 2582 flags.go:64] FLAG: --read-only-port="10255" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092686 2582 flags.go:64] FLAG: --register-node="true" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092689 2582 flags.go:64] FLAG: --register-with-taints="" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092693 2582 flags.go:64] FLAG: --registry-burst="10" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092699 2582 flags.go:64] FLAG: --registry-qps="5" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092702 2582 flags.go:64] FLAG: --reserved-cpus="" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092705 2582 flags.go:64] FLAG: --reserved-memory="" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092709 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092712 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092715 2582 flags.go:64] FLAG: --rotate-certificates="false" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092718 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092721 2582 flags.go:64] FLAG: --runonce="false" Jun 10 12:36:50.100317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092723 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092726 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092729 2582 flags.go:64] FLAG: --seccomp-default="false" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092732 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092735 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092737 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092740 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092743 2582 flags.go:64] FLAG: --storage-driver-password="root" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092746 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092749 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092751 2582 flags.go:64] FLAG: --storage-driver-user="root" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092754 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092758 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092761 2582 flags.go:64] FLAG: --system-cgroups="" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092764 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092769 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092771 2582 flags.go:64] FLAG: --tls-cert-file="" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092774 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092778 2582 flags.go:64] FLAG: --tls-min-version="" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092781 2582 flags.go:64] FLAG: --tls-private-key-file="" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092783 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092786 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092789 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092792 2582 flags.go:64] FLAG: --v="2" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092796 2582 flags.go:64] FLAG: --version="false" Jun 10 12:36:50.100938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092801 2582 flags.go:64] FLAG: --vmodule="" Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092804 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.092809 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092933 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092938 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092941 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092944 2582 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092947 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092949 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092951 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092954 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092957 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092959 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092961 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092964 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092966 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092969 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092972 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092974 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092979 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:36:50.101668 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092981 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092984 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092986 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092989 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092991 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092994 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092996 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.092999 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093001 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093003 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093006 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093008 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093010 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093013 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093018 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093022 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093025 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093028 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093030 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093033 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:36:50.102237 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093036 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093039 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093041 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093044 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093046 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093049 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093051 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093054 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093056 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093059 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093061 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093066 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093070 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093072 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093075 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093077 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093080 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093082 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093086 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:36:50.102726 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093089 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093092 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093110 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093112 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093114 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093117 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093119 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093124 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093127 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093130 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093132 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093135 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093137 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093140 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093142 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093145 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093147 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093150 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093152 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:36:50.103214 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093154 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093157 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093159 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093162 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093164 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093169 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093172 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093174 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093177 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093179 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093182 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093184 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093187 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093190 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093192 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093194 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093197 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093199 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093202 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093204 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:36:50.103685 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093208 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093214 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093217 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.093220 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.093703 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.100266 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.100281 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.100301 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.100307 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100363 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100369 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100373 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100376 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100379 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100381 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:36:50.104205 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100384 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100387 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100390 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100392 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100395 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100398 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100401 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100403 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100406 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100408 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100411 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100413 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100416 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100418 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100421 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100423 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100426 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100430 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100433 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100436 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:36:50.104582 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100438 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100441 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100444 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100448 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100451 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100454 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100456 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100459 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100461 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100464 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100467 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100469 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100472 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100474 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100476 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100479 2582 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100481 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100484 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100486 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:36:50.105119 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100489 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100493 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100496 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100499 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100503 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100506 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100509 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100512 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100515 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100517 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100520 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100523 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100525 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100528 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100530 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100533 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100536 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100539 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100541 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:36:50.105600 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100544 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100546 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100549 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100551 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100553 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100556 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100558 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100561 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100563 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100566 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100568 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100570 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100573 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100575 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100577 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100580 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100582 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100585 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100587 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100590 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:36:50.106277 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100593 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100595 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100598 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100600 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100603 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100605 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100607 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100610 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100612 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100615 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100618 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100621 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100623 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100626 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100628 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:36:50.106808 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.100633 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100750 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100755 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100758 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100761 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100764 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100767 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100770 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100774 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100777 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100780 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100783 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100785 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100788 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100791 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100793 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100796 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100799 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100802 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100804 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:36:50.107260 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100807 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100809 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100812 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100814 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100816 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100819 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100822 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100825 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100827 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100830 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100833 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100835 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100838 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100840 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100843 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100845 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100847 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100850 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100852 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100854 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:36:50.107748 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100857 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100859 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100861 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100864 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100866 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100869 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100871 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100873 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100876 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100879 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100881 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100884 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100886 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100890 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100893 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100896 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100898 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100900 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:36:50.108342 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100904 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100907 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100910 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100913 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100916 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100919 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100922 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100924 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100927 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100929 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100931 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100934 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100939 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100942 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100944 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100947 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100949 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100952 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100955 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:36:50.109031 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100957 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100960 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100962 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100965 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100967 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100970 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100972 2582 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100974 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100977 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100979 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100981 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100984 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100986 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100989 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100992 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100994 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100996 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.100999 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.101002 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.101005 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:36:50.109655 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.101007 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:36:50.110297 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.101010 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:36:50.110297 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:50.101012 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:36:50.110297 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.101016 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 12:36:50.110297 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.101144 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jun 10 12:36:50.110297 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.104480 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 10 12:36:50.110297 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.105920 2582 server.go:1019] "Starting client certificate rotation" Jun 10 12:36:50.110297 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.106038 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 12:36:50.110297 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.106108 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 12:36:50.110297 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.109454 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 12:36:50.110725 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.110693 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 12:36:50.122975 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.122954 2582 log.go:25] "Validated CRI v1 runtime API" Jun 10 12:36:50.127612 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.127590 2582 log.go:25] "Validated CRI v1 image API" Jun 10 12:36:50.128512 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.128494 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 10 12:36:50.130665 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.130641 2582 fs.go:135] Filesystem UUIDs: map[219fdc60-4ece-4f90-a286-33472c2057ad:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9db2369a-919d-4ad8-895c-27d8bbd96cad:/dev/nvme0n1p3] Jun 10 12:36:50.130746 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.130660 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 10 12:36:50.136300 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.136180 2582 manager.go:216] Machine: {Timestamp:2026-06-10 12:36:50.135005962 +0000 UTC m=+0.345851940 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099420 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cba003612e5055f7e98b996657f34 SystemUUID:ec2cba00-3612-e505-5f7e-98b996657f34 BootID:5ce694ee-77f3-4bce-bc42-a8da6fca82e9 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:12:5e:51:a6:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:12:5e:51:a6:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:0d:3f:78:85:bf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 10 12:36:50.136300 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.136289 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 10 12:36:50.136419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.136407 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 10 12:36:50.138869 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.138839 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 10 12:36:50.139014 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.138873 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-99.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 10 12:36:50.139818 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.139805 2582 topology_manager.go:138] "Creating topology manager with none policy" Jun 10 12:36:50.139861 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.139822 2582 container_manager_linux.go:309] "Creating device plugin manager" Jun 10 12:36:50.139861 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.139831 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 12:36:50.140576 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.140565 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 12:36:50.141353 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.141344 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 10 12:36:50.142669 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.142654 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 12:36:50.144634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.144621 2582 state_mem.go:36] "Initialized new in-memory state store" Jun 10 12:36:50.144754 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.144745 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 10 12:36:50.144788 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.144777 2582 kubelet.go:486] "Attempting to sync node with API server" Jun 10 12:36:50.144816 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.144790 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 10 12:36:50.144846 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.144814 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 10 12:36:50.144846 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.144828 2582 kubelet.go:398] "Adding apiserver pod source" Jun 10 12:36:50.144846 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.144840 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 10 12:36:50.145224 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.145213 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 12:36:50.145287 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.145231 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 12:36:50.148828 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.148812 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 10 12:36:50.150086 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.150073 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 10 12:36:50.150151 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.150122 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 10 12:36:50.151141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151120 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 10 12:36:50.151141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151139 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 10 12:36:50.151141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151146 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 10 12:36:50.151329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151152 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 10 12:36:50.151329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151161 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 10 12:36:50.151329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151166 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 10 12:36:50.151329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151174 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 10 12:36:50.151329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151179 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 10 12:36:50.151329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151186 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 10 12:36:50.151329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151191 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 10 12:36:50.151329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151200 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 10 12:36:50.151329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.151205 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 10 12:36:50.152700 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.152688 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 10 12:36:50.152700 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.152700 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 10 12:36:50.159980 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.155776 2582 server.go:1268] "Started kubelet" Jun 10 12:36:50.159980 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.156582 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 10 12:36:50.159980 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.158825 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 10 12:36:50.159980 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.158862 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 10 12:36:50.159980 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.159181 2582 server.go:310] "Adding debug handlers to kubelet server" Jun 10 12:36:50.156512 ip-10-0-138-99 systemd[1]: Started Kubernetes Kubelet. Jun 10 12:36:50.160857 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.160121 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-99.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 10 12:36:50.160857 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.160743 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 10 12:36:50.163282 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.163264 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kntpc" Jun 10 12:36:50.166155 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.166137 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 10 12:36:50.168846 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.168827 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 10 12:36:50.168846 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.168840 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 10 12:36:50.169004 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.168954 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:50.169589 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.169539 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 10 12:36:50.169761 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.169739 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 10 12:36:50.170374 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170357 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 10 12:36:50.170374 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170376 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 10 12:36:50.170507 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.170384 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 10 12:36:50.170507 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170475 2582 reconstruct.go:97] "Volume reconstruction finished" Jun 10 12:36:50.170507 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170484 2582 reconciler.go:29] "Reconciler: start to sync state" Jun 10 12:36:50.170685 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170645 2582 factory.go:153] Registering CRI-O factory Jun 10 12:36:50.170685 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170658 2582 factory.go:223] Registration of the crio container factory successfully Jun 10 12:36:50.170761 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.169504 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-99.ec2.internal.18b7b8799f8df6c6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-99.ec2.internal,UID:ip-10-0-138-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-99.ec2.internal,},FirstTimestamp:2026-06-10 12:36:50.155755206 +0000 UTC m=+0.366601183,LastTimestamp:2026-06-10 12:36:50.155755206 +0000 UTC m=+0.366601183,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-99.ec2.internal,}" Jun 10 12:36:50.170761 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170701 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 10 12:36:50.170761 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170712 2582 factory.go:55] Registering systemd factory Jun 10 12:36:50.170761 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170721 2582 factory.go:223] Registration of the systemd container factory successfully Jun 10 12:36:50.170761 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170744 2582 factory.go:103] Registering Raw factory Jun 10 12:36:50.170761 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.170758 2582 manager.go:1196] Started watching for new ooms in manager Jun 10 12:36:50.171188 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.171174 2582 manager.go:349] Starting recovery of all containers Jun 10 12:36:50.172389 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.172296 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 10 12:36:50.172389 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.172429 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 10 12:36:50.177394 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.177369 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kntpc" Jun 10 12:36:50.181686 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.181664 2582 manager.go:354] Recovery completed Jun 10 12:36:50.186799 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.186785 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:36:50.188906 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.188890 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:36:50.188973 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.188923 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:36:50.188973 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.188940 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:36:50.189587 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.189572 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 10 12:36:50.189636 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.189589 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 10 12:36:50.189636 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.189607 2582 state_mem.go:36] "Initialized new in-memory state store" Jun 10 12:36:50.189636 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.189626 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:36:50.190408 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.190393 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:36:50.190452 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.190420 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:36:50.190452 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.190429 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:36:50.190880 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.190868 2582 policy_none.go:49] "None policy: Start" Jun 10 12:36:50.190911 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.190885 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 10 12:36:50.190911 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.190895 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 10 12:36:50.191488 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.191423 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-99.ec2.internal.18b7b879a187cf67 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-99.ec2.internal,UID:ip-10-0-138-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-99.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-99.ec2.internal,},FirstTimestamp:2026-06-10 12:36:50.188906343 +0000 UTC m=+0.399752324,LastTimestamp:2026-06-10 12:36:50.188906343 +0000 UTC m=+0.399752324,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-99.ec2.internal,}" Jun 10 12:36:50.191882 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.191862 2582 policy_none.go:47] "Start" Jun 10 12:36:50.231699 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.231679 2582 manager.go:337] "Starting Device Plugin manager" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.231759 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.231774 2582 server.go:87] "Starting device plugin registration server" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.232049 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.232062 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.232177 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.232306 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.232316 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.233110 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.233154 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:50.240165 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.239663 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 10 12:36:50.241053 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.241033 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 10 12:36:50.241175 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.241070 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 10 12:36:50.241175 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.241091 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jun 10 12:36:50.241280 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.241200 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 10 12:36:50.243472 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.243449 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:36:50.333072 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.332979 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:36:50.337383 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.337357 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:36:50.337494 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.337394 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:36:50.337494 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.337408 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:36:50.337494 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.337441 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.342277 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.342250 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal"] Jun 10 12:36:50.342329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.342320 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:36:50.345658 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.345643 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.345708 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.345665 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-99.ec2.internal\": node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:50.346440 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.346423 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:36:50.346520 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.346453 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:36:50.346520 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.346464 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:36:50.347732 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.347719 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:36:50.347844 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.347829 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.347880 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.347860 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:36:50.348951 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.348922 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:36:50.349055 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.348933 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:36:50.349055 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.348985 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:36:50.349055 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.348954 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:36:50.349055 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.349014 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:36:50.349055 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.349034 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:36:50.350396 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.350380 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.350482 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.350409 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:36:50.351135 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.351121 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:36:50.351189 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.351148 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:36:50.351189 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.351159 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:36:50.368619 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.368595 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:50.371733 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.371705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cdfca2d07c4ff59b8cccc8704407fc86-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"cdfca2d07c4ff59b8cccc8704407fc86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.371819 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.371747 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdfca2d07c4ff59b8cccc8704407fc86-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"cdfca2d07c4ff59b8cccc8704407fc86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.371819 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.371764 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89d633427a813ac026468ec864f4e82d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-99.ec2.internal\" (UID: \"89d633427a813ac026468ec864f4e82d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.381967 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.381940 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-99.ec2.internal\" not found" node="ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.386418 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.386401 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-99.ec2.internal\" not found" node="ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.469615 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.469578 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:50.472850 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.472829 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cdfca2d07c4ff59b8cccc8704407fc86-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"cdfca2d07c4ff59b8cccc8704407fc86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.472904 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.472860 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdfca2d07c4ff59b8cccc8704407fc86-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"cdfca2d07c4ff59b8cccc8704407fc86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.472904 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.472875 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89d633427a813ac026468ec864f4e82d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-99.ec2.internal\" (UID: \"89d633427a813ac026468ec864f4e82d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.472980 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.472912 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89d633427a813ac026468ec864f4e82d-config\") pod \"kube-apiserver-proxy-ip-10-0-138-99.ec2.internal\" (UID: \"89d633427a813ac026468ec864f4e82d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.472980 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.472933 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cdfca2d07c4ff59b8cccc8704407fc86-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"cdfca2d07c4ff59b8cccc8704407fc86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.472980 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.472948 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdfca2d07c4ff59b8cccc8704407fc86-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal\" (UID: \"cdfca2d07c4ff59b8cccc8704407fc86\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.570649 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.570616 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:50.671482 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.671413 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:50.685671 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.685639 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.691769 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:50.691732 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" Jun 10 12:36:50.771808 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.771772 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:50.872297 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.872269 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:50.972921 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:50.972833 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:51.073393 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:51.073360 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:51.105696 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.105666 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 10 12:36:51.106366 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.105840 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 10 12:36:51.174159 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:51.174133 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:51.180544 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.180513 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-09 12:31:50 +0000 UTC" deadline="2028-02-25 01:42:35.950968154 +0000 UTC" Jun 10 12:36:51.180544 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.180543 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14989h5m44.770429226s" Jun 10 12:36:51.201406 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:51.201236 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89d633427a813ac026468ec864f4e82d.slice/crio-b38d64d32dae66961d160969189d638d3f866d757b4f2c2a367335032973b139 WatchSource:0}: Error finding container b38d64d32dae66961d160969189d638d3f866d757b4f2c2a367335032973b139: Status 404 returned error can't find the container with id b38d64d32dae66961d160969189d638d3f866d757b4f2c2a367335032973b139 Jun 10 12:36:51.201758 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:51.201732 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdfca2d07c4ff59b8cccc8704407fc86.slice/crio-f47c4f190aa2103b27d51eac673f075a74f5e46f675db7eda85e953545041af6 WatchSource:0}: Error finding container f47c4f190aa2103b27d51eac673f075a74f5e46f675db7eda85e953545041af6: Status 404 returned error can't find the container with id f47c4f190aa2103b27d51eac673f075a74f5e46f675db7eda85e953545041af6 Jun 10 12:36:51.207218 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.207203 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 12:36:51.244231 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.244088 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" event={"ID":"89d633427a813ac026468ec864f4e82d","Type":"ContainerStarted","Data":"b38d64d32dae66961d160969189d638d3f866d757b4f2c2a367335032973b139"} Jun 10 12:36:51.245134 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.245084 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" event={"ID":"cdfca2d07c4ff59b8cccc8704407fc86","Type":"ContainerStarted","Data":"f47c4f190aa2103b27d51eac673f075a74f5e46f675db7eda85e953545041af6"} Jun 10 12:36:51.274524 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:51.274493 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-99.ec2.internal\" not found" Jun 10 12:36:51.320368 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.320345 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:36:51.369357 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.369327 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" Jun 10 12:36:51.382049 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.382024 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 12:36:51.383767 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.383751 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" Jun 10 12:36:51.391543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.391524 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 12:36:51.521264 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.521183 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:36:51.533989 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:51.533958 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:36:52.128304 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.128266 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:36:52.146540 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.146506 2582 apiserver.go:52] "Watching apiserver" Jun 10 12:36:52.155697 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.155669 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 10 12:36:52.156113 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.156063 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-tfsmb","kube-system/konnectivity-agent-99zc2","kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km","openshift-cluster-node-tuning-operator/tuned-8s6gl","openshift-dns/node-resolver-5m4pd","openshift-image-registry/node-ca-gqpm5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal","openshift-multus/multus-additional-cni-plugins-z7vkn","kube-system/global-pull-secret-syncer-fww7p","openshift-multus/multus-jzvbs","openshift-multus/network-metrics-daemon-zlgzd","openshift-network-diagnostics/network-check-target-pf78n","openshift-network-operator/iptables-alerter-nff49"] Jun 10 12:36:52.157639 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.157615 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.158958 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.158939 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:52.159051 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.159011 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:36:52.160013 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.159990 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.160455 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.160436 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 10 12:36:52.160532 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.160492 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 10 12:36:52.160745 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.160727 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pq6rb\"" Jun 10 12:36:52.160845 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.160776 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 10 12:36:52.161329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.161312 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.162532 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.162514 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.162917 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.162880 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 10 12:36:52.163390 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.163371 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:36:52.163714 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.163698 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 10 12:36:52.163868 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.163853 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.164685 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.164455 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 10 12:36:52.164685 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.164498 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 10 12:36:52.164685 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.164457 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 10 12:36:52.164685 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.164505 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ts4rk\"" Jun 10 12:36:52.165189 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.165169 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.166499 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.165856 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:36:52.166499 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.165942 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qkf89\"" Jun 10 12:36:52.166499 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.165984 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 10 12:36:52.166599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.166503 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 10 12:36:52.166799 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.166783 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.166831 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.166808 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 10 12:36:52.167408 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.167393 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9h6sl\"" Jun 10 12:36:52.167893 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.167869 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:36:52.168362 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.168331 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 10 12:36:52.168575 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.168555 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 10 12:36:52.169509 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.169240 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 10 12:36:52.169509 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.169280 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wjggs\"" Jun 10 12:36:52.169509 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.169357 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 10 12:36:52.169509 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.169377 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.169509 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.169418 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 10 12:36:52.169509 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.169443 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:36:52.170450 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.170432 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-g4j6c\"" Jun 10 12:36:52.170568 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.170550 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 10 12:36:52.170715 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.170694 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 10 12:36:52.170791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.170751 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 10 12:36:52.170923 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.170908 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sv8wn\"" Jun 10 12:36:52.170983 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.170966 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.172605 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.172554 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 10 12:36:52.172969 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.172948 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:52.173054 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.172976 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 10 12:36:52.173237 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.173181 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:36:52.174970 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.173361 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 10 12:36:52.174970 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.173392 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 10 12:36:52.179707 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.179526 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 10 12:36:52.179707 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.179649 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v4sl6\"" Jun 10 12:36:52.179884 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.179870 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 10 12:36:52.182136 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182114 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-os-release\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.182236 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182153 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-tuned\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.182236 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182179 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aa02a27-aa46-4bb2-b823-a91092066a54-cni-binary-copy\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.182236 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182203 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-ovnkube-script-lib\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.182392 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-cni-binary-copy\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.182392 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182287 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d022e905-ba12-4217-8018-186dd3865aa5-host-slash\") pod \"iptables-alerter-nff49\" (UID: \"d022e905-ba12-4217-8018-186dd3865aa5\") " pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.182392 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182311 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-kubernetes\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.182392 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182332 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-host\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.182392 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182357 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.182640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182406 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ghg4\" (UniqueName: \"kubernetes.io/projected/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-kube-api-access-2ghg4\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.182640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182448 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjw4j\" (UniqueName: \"kubernetes.io/projected/8aa02a27-aa46-4bb2-b823-a91092066a54-kube-api-access-zjw4j\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.182640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182478 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-cni-bin\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.182640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182501 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-cni-netd\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.182640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182524 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-ovnkube-config\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.182640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182563 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af9f4f53-a664-4c7d-a457-9e0779ad3c16-host\") pod \"node-ca-gqpm5\" (UID: \"af9f4f53-a664-4c7d-a457-9e0779ad3c16\") " pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.182640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182597 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-socket-dir-parent\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.182640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182625 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d022e905-ba12-4217-8018-186dd3865aa5-iptables-alerter-script\") pod \"iptables-alerter-nff49\" (UID: \"d022e905-ba12-4217-8018-186dd3865aa5\") " pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182648 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccfbq\" (UniqueName: \"kubernetes.io/projected/d022e905-ba12-4217-8018-186dd3865aa5-kube-api-access-ccfbq\") pod \"iptables-alerter-nff49\" (UID: \"d022e905-ba12-4217-8018-186dd3865aa5\") " pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182670 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-modprobe-d\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182696 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-kubelet\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182718 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-ovn-node-metrics-cert\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182851 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-run-netns\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182901 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-socket-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182962 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-device-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182977 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-sysconfig\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.182991 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-cnibin\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183004 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-node-log\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183030 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9303f166-88cd-4695-8a92-ead714e32ca1-agent-certs\") pod \"konnectivity-agent-99zc2\" (UID: \"9303f166-88cd-4695-8a92-ead714e32ca1\") " pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183055 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:52.183069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183072 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-conf-dir\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183088 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-os-release\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183141 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/af9f4f53-a664-4c7d-a457-9e0779ad3c16-serviceca\") pod \"node-ca-gqpm5\" (UID: \"af9f4f53-a664-4c7d-a457-9e0779ad3c16\") " pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183165 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfg5h\" (UniqueName: \"kubernetes.io/projected/666e8e25-9dde-4db2-a3e2-3e75b5e24059-kube-api-access-jfg5h\") pod \"node-resolver-5m4pd\" (UID: \"666e8e25-9dde-4db2-a3e2-3e75b5e24059\") " pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183193 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-var-lib-cni-multus\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-etc-selinux\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183255 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtxhn\" (UniqueName: \"kubernetes.io/projected/c95dae07-0e6a-4996-b135-98cff030bfa8-kube-api-access-mtxhn\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183275 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-env-overrides\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183296 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9303f166-88cd-4695-8a92-ead714e32ca1-konnectivity-ca\") pod \"konnectivity-agent-99zc2\" (UID: \"9303f166-88cd-4695-8a92-ead714e32ca1\") " pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183310 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-cni-dir\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183324 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-cnibin\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183361 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8aa02a27-aa46-4bb2-b823-a91092066a54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183387 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-run-netns\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183406 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-run-ovn\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183428 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-system-cni-dir\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183445 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-hostroot\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.183803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183458 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-registration-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183497 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-run-systemd\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183526 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/63acd877-6555-4895-8e27-9b7540f95de8-kubelet-config\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183560 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-etc-kubernetes\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183604 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-sys-fs\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183631 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-sysctl-conf\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183654 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-lib-modules\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183677 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-systemd-units\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183700 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-log-socket\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183739 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183768 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-sysctl-d\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183803 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-systemd\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183826 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-run-ovn-kubernetes\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183876 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-run\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-sys\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183920 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-var-lib-kubelet\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.184534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/666e8e25-9dde-4db2-a3e2-3e75b5e24059-tmp-dir\") pod \"node-resolver-5m4pd\" (UID: \"666e8e25-9dde-4db2-a3e2-3e75b5e24059\") " pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183967 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.183991 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-var-lib-openvswitch\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184015 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nbf5x\"" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184015 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-run-openvswitch\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184072 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbrz6\" (UniqueName: \"kubernetes.io/projected/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-kube-api-access-zbrz6\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184113 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-run-k8s-cni-cncf-io\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184137 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-var-lib-cni-bin\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184160 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-daemon-config\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184184 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-tmp\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184206 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-system-cni-dir\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184229 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184250 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/63acd877-6555-4895-8e27-9b7540f95de8-dbus\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184272 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-var-lib-kubelet\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184295 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjkv2\" (UniqueName: \"kubernetes.io/projected/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-kube-api-access-qjkv2\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184314 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8aa02a27-aa46-4bb2-b823-a91092066a54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184351 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-slash\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.185164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184403 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-etc-openvswitch\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.185858 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184428 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-run-multus-certs\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.185858 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqb8b\" (UniqueName: \"kubernetes.io/projected/af9f4f53-a664-4c7d-a457-9e0779ad3c16-kube-api-access-gqb8b\") pod \"node-ca-gqpm5\" (UID: \"af9f4f53-a664-4c7d-a457-9e0779ad3c16\") " pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.185858 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.184480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/666e8e25-9dde-4db2-a3e2-3e75b5e24059-hosts-file\") pod \"node-resolver-5m4pd\" (UID: \"666e8e25-9dde-4db2-a3e2-3e75b5e24059\") " pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.202930 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.202905 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 12:36:52.240627 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.240596 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ffsng" Jun 10 12:36:52.249251 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.249222 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ffsng" Jun 10 12:36:52.271555 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.271519 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 10 12:36:52.284900 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.284870 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-modprobe-d\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.284900 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.284905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-kubelet\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.284930 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-ovn-node-metrics-cert\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.284954 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-run-netns\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.284975 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-socket-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.284989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-device-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285007 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-sysconfig\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285031 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-cnibin\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285056 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-node-log\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285002 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-kubelet\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9303f166-88cd-4695-8a92-ead714e32ca1-agent-certs\") pod \"konnectivity-agent-99zc2\" (UID: \"9303f166-88cd-4695-8a92-ead714e32ca1\") " pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285064 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-modprobe-d\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285121 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285127 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-device-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285148 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285152 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-cnibin\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.285161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-conf-dir\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-os-release\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285208 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/af9f4f53-a664-4c7d-a457-9e0779ad3c16-serviceca\") pod \"node-ca-gqpm5\" (UID: \"af9f4f53-a664-4c7d-a457-9e0779ad3c16\") " pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285210 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-sysconfig\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285224 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfg5h\" (UniqueName: \"kubernetes.io/projected/666e8e25-9dde-4db2-a3e2-3e75b5e24059-kube-api-access-jfg5h\") pod \"node-resolver-5m4pd\" (UID: \"666e8e25-9dde-4db2-a3e2-3e75b5e24059\") " pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285236 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-run-netns\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285246 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-var-lib-cni-multus\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285266 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-etc-selinux\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285287 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtxhn\" (UniqueName: \"kubernetes.io/projected/c95dae07-0e6a-4996-b135-98cff030bfa8-kube-api-access-mtxhn\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285303 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-env-overrides\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9303f166-88cd-4695-8a92-ead714e32ca1-konnectivity-ca\") pod \"konnectivity-agent-99zc2\" (UID: \"9303f166-88cd-4695-8a92-ead714e32ca1\") " pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285337 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-cni-dir\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285358 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-cnibin\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8aa02a27-aa46-4bb2-b823-a91092066a54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285397 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-run-netns\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285410 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-run-ovn\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-system-cni-dir\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.285827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285443 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-hostroot\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285461 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-registration-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285495 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-run-systemd\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285502 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-node-log\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285525 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/63acd877-6555-4895-8e27-9b7540f95de8-kubelet-config\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285529 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285594 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/63acd877-6555-4895-8e27-9b7540f95de8-kubelet-config\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285646 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-conf-dir\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285719 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-os-release\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285823 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-socket-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285879 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-cnibin\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.285947 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-system-cni-dir\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286193 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/af9f4f53-a664-4c7d-a457-9e0779ad3c16-serviceca\") pod \"node-ca-gqpm5\" (UID: \"af9f4f53-a664-4c7d-a457-9e0779ad3c16\") " pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286289 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-etc-kubernetes\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-sys-fs\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-sysctl-conf\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286354 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-env-overrides\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-lib-modules\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.286624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286403 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-systemd-units\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286428 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-log-socket\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286447 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8aa02a27-aa46-4bb2-b823-a91092066a54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286451 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-hostroot\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286454 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286488 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-sysctl-d\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286505 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286512 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-systemd\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286532 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-run-ovn-kubernetes\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286545 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-run-netns\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286550 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-run-ovn\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-run\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286580 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-sys\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286596 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-var-lib-cni-multus\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-var-lib-kubelet\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286636 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-run-ovn-kubernetes\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286649 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-sysctl-conf\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.287471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286632 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/666e8e25-9dde-4db2-a3e2-3e75b5e24059-tmp-dir\") pod \"node-resolver-5m4pd\" (UID: \"666e8e25-9dde-4db2-a3e2-3e75b5e24059\") " pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286689 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-registration-dir\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286697 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286723 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-var-lib-openvswitch\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286741 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-cni-dir\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286745 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-lib-modules\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286789 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-systemd\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286794 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-run\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286835 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-log-socket\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286839 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9303f166-88cd-4695-8a92-ead714e32ca1-konnectivity-ca\") pod \"konnectivity-agent-99zc2\" (UID: \"9303f166-88cd-4695-8a92-ead714e32ca1\") " pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286861 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-sysctl-d\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286891 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-etc-selinux\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286747 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-run-openvswitch\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286904 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-var-lib-openvswitch\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286836 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c95dae07-0e6a-4996-b135-98cff030bfa8-sys-fs\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286925 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbrz6\" (UniqueName: \"kubernetes.io/projected/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-kube-api-access-zbrz6\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286944 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-systemd-units\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.286944 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:52.288268 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286950 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-run-k8s-cni-cncf-io\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286505 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-run-systemd\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286975 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-var-lib-cni-bin\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.287004 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret podName:63acd877-6555-4895-8e27-9b7540f95de8 nodeName:}" failed. No retries permitted until 2026-06-10 12:36:52.786988179 +0000 UTC m=+2.997834165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret") pod "global-pull-secret-syncer-fww7p" (UID: "63acd877-6555-4895-8e27-9b7540f95de8") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-daemon-config\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287022 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-sys\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287019 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-var-lib-cni-bin\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287058 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-tmp\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287067 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-etc-kubernetes\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.286863 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-run-openvswitch\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287089 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-system-cni-dir\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287137 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287145 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/666e8e25-9dde-4db2-a3e2-3e75b5e24059-tmp-dir\") pod \"node-resolver-5m4pd\" (UID: \"666e8e25-9dde-4db2-a3e2-3e75b5e24059\") " pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287171 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th45f\" (UniqueName: \"kubernetes.io/projected/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-kube-api-access-th45f\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287191 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-var-lib-kubelet\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287197 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/63acd877-6555-4895-8e27-9b7540f95de8-dbus\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287221 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-var-lib-kubelet\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.289221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287244 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjkv2\" (UniqueName: \"kubernetes.io/projected/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-kube-api-access-qjkv2\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287267 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8aa02a27-aa46-4bb2-b823-a91092066a54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287289 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-slash\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-etc-openvswitch\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287313 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-run-k8s-cni-cncf-io\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287328 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-run-multus-certs\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287353 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqb8b\" (UniqueName: \"kubernetes.io/projected/af9f4f53-a664-4c7d-a457-9e0779ad3c16-kube-api-access-gqb8b\") pod \"node-ca-gqpm5\" (UID: \"af9f4f53-a664-4c7d-a457-9e0779ad3c16\") " pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287361 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-var-lib-kubelet\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287377 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/666e8e25-9dde-4db2-a3e2-3e75b5e24059-hosts-file\") pod \"node-resolver-5m4pd\" (UID: \"666e8e25-9dde-4db2-a3e2-3e75b5e24059\") " pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287426 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-os-release\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287448 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-tuned\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287464 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aa02a27-aa46-4bb2-b823-a91092066a54-cni-binary-copy\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287506 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-ovnkube-script-lib\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287514 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287531 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-cni-binary-copy\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287557 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d022e905-ba12-4217-8018-186dd3865aa5-host-slash\") pod \"iptables-alerter-nff49\" (UID: \"d022e905-ba12-4217-8018-186dd3865aa5\") " pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287673 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/63acd877-6555-4895-8e27-9b7540f95de8-dbus\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.290149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287679 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-daemon-config\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287714 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-kubernetes\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-host\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287769 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287776 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-slash\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287795 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ghg4\" (UniqueName: \"kubernetes.io/projected/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-kube-api-access-2ghg4\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287814 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-etc-openvswitch\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjw4j\" (UniqueName: \"kubernetes.io/projected/8aa02a27-aa46-4bb2-b823-a91092066a54-kube-api-access-zjw4j\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287840 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-host-run-multus-certs\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287849 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-cni-bin\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287874 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-cni-netd\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287743 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d022e905-ba12-4217-8018-186dd3865aa5-host-slash\") pod \"iptables-alerter-nff49\" (UID: \"d022e905-ba12-4217-8018-186dd3865aa5\") " pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.287904 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-ovnkube-config\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288068 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/666e8e25-9dde-4db2-a3e2-3e75b5e24059-hosts-file\") pod \"node-resolver-5m4pd\" (UID: \"666e8e25-9dde-4db2-a3e2-3e75b5e24059\") " pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288264 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-ovnkube-script-lib\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288279 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-kubernetes\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288327 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-host\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288385 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-cni-bin\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288389 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-os-release\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288419 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aa02a27-aa46-4bb2-b823-a91092066a54-system-cni-dir\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288439 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-cni-netd\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288534 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af9f4f53-a664-4c7d-a457-9e0779ad3c16-host\") pod \"node-ca-gqpm5\" (UID: \"af9f4f53-a664-4c7d-a457-9e0779ad3c16\") " pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-socket-dir-parent\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288589 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d022e905-ba12-4217-8018-186dd3865aa5-iptables-alerter-script\") pod \"iptables-alerter-nff49\" (UID: \"d022e905-ba12-4217-8018-186dd3865aa5\") " pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccfbq\" (UniqueName: \"kubernetes.io/projected/d022e905-ba12-4217-8018-186dd3865aa5-kube-api-access-ccfbq\") pod \"iptables-alerter-nff49\" (UID: \"d022e905-ba12-4217-8018-186dd3865aa5\") " pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288751 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-cni-binary-copy\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288879 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8aa02a27-aa46-4bb2-b823-a91092066a54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288941 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.288944 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af9f4f53-a664-4c7d-a457-9e0779ad3c16-host\") pod \"node-ca-gqpm5\" (UID: \"af9f4f53-a664-4c7d-a457-9e0779ad3c16\") " pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.289013 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-multus-socket-dir-parent\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.289270 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-ovnkube-config\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.289417 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aa02a27-aa46-4bb2-b823-a91092066a54-cni-binary-copy\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.289697 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d022e905-ba12-4217-8018-186dd3865aa5-iptables-alerter-script\") pod \"iptables-alerter-nff49\" (UID: \"d022e905-ba12-4217-8018-186dd3865aa5\") " pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.289729 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-tmp\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.290525 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9303f166-88cd-4695-8a92-ead714e32ca1-agent-certs\") pod \"konnectivity-agent-99zc2\" (UID: \"9303f166-88cd-4695-8a92-ead714e32ca1\") " pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:36:52.291954 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.290715 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-etc-tuned\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.292666 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.292384 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-ovn-node-metrics-cert\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.307072 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.307014 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccfbq\" (UniqueName: \"kubernetes.io/projected/d022e905-ba12-4217-8018-186dd3865aa5-kube-api-access-ccfbq\") pod \"iptables-alerter-nff49\" (UID: \"d022e905-ba12-4217-8018-186dd3865aa5\") " pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.308050 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.307757 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:36:52.308050 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.307782 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:36:52.308050 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.307795 2582 projected.go:196] Error preparing data for projected volume kube-api-access-52fr9 for pod openshift-network-diagnostics/network-check-target-pf78n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:52.308050 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.307854 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9 podName:491acc63-2d58-4cca-bfbd-e5d0a0adae9a nodeName:}" failed. No retries permitted until 2026-06-10 12:36:52.807835141 +0000 UTC m=+3.018681126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-52fr9" (UniqueName: "kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9") pod "network-check-target-pf78n" (UID: "491acc63-2d58-4cca-bfbd-e5d0a0adae9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:52.308348 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.308148 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjkv2\" (UniqueName: \"kubernetes.io/projected/fb3dc5bb-5a81-4b35-8a0c-7c9889737bef-kube-api-access-qjkv2\") pod \"tuned-8s6gl\" (UID: \"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef\") " pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.308837 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.308810 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqb8b\" (UniqueName: \"kubernetes.io/projected/af9f4f53-a664-4c7d-a457-9e0779ad3c16-kube-api-access-gqb8b\") pod \"node-ca-gqpm5\" (UID: \"af9f4f53-a664-4c7d-a457-9e0779ad3c16\") " pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.309185 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.309163 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtxhn\" (UniqueName: \"kubernetes.io/projected/c95dae07-0e6a-4996-b135-98cff030bfa8-kube-api-access-mtxhn\") pod \"aws-ebs-csi-driver-node-x66km\" (UID: \"c95dae07-0e6a-4996-b135-98cff030bfa8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.309275 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.309252 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ghg4\" (UniqueName: \"kubernetes.io/projected/bd979c87-5eb1-4cef-a302-4ab9570eb4cd-kube-api-access-2ghg4\") pod \"multus-jzvbs\" (UID: \"bd979c87-5eb1-4cef-a302-4ab9570eb4cd\") " pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.309688 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.309648 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjw4j\" (UniqueName: \"kubernetes.io/projected/8aa02a27-aa46-4bb2-b823-a91092066a54-kube-api-access-zjw4j\") pod \"multus-additional-cni-plugins-z7vkn\" (UID: \"8aa02a27-aa46-4bb2-b823-a91092066a54\") " pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.309827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.309806 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfg5h\" (UniqueName: \"kubernetes.io/projected/666e8e25-9dde-4db2-a3e2-3e75b5e24059-kube-api-access-jfg5h\") pod \"node-resolver-5m4pd\" (UID: \"666e8e25-9dde-4db2-a3e2-3e75b5e24059\") " pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.309907 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.309886 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbrz6\" (UniqueName: \"kubernetes.io/projected/ee00a67b-e3d6-4c47-81dc-d5a523e04dc7-kube-api-access-zbrz6\") pod \"ovnkube-node-tfsmb\" (UID: \"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7\") " pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.389204 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.389085 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th45f\" (UniqueName: \"kubernetes.io/projected/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-kube-api-access-th45f\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:52.389204 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.389177 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:52.389404 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.389302 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:36:52.389439 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.389411 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:36:52.889390455 +0000 UTC m=+3.100236423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:36:52.404137 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.404113 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th45f\" (UniqueName: \"kubernetes.io/projected/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-kube-api-access-th45f\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:52.471355 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.471319 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gqpm5" Jun 10 12:36:52.479434 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.479408 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nff49" Jun 10 12:36:52.487690 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.487664 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" Jun 10 12:36:52.492691 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.492666 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" Jun 10 12:36:52.499886 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.499863 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5m4pd" Jun 10 12:36:52.506520 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.506502 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" Jun 10 12:36:52.514186 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.514164 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:36:52.519283 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.519251 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:36:52.523840 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.523823 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jzvbs" Jun 10 12:36:52.791678 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.791601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:52.791833 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.791770 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:52.791894 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.791851 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret podName:63acd877-6555-4895-8e27-9b7540f95de8 nodeName:}" failed. No retries permitted until 2026-06-10 12:36:53.791830412 +0000 UTC m=+4.002676394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret") pod "global-pull-secret-syncer-fww7p" (UID: "63acd877-6555-4895-8e27-9b7540f95de8") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:52.892157 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.892115 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:52.892316 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:52.892168 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:52.892316 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.892219 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:36:52.892316 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.892287 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:36:52.892316 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.892304 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:36:52.892316 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.892316 2582 projected.go:196] Error preparing data for projected volume kube-api-access-52fr9 for pod openshift-network-diagnostics/network-check-target-pf78n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:52.892532 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.892292 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:36:53.89227574 +0000 UTC m=+4.103121707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:36:52.892532 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:52.892382 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9 podName:491acc63-2d58-4cca-bfbd-e5d0a0adae9a nodeName:}" failed. No retries permitted until 2026-06-10 12:36:53.892364689 +0000 UTC m=+4.103210660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-52fr9" (UniqueName: "kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9") pod "network-check-target-pf78n" (UID: "491acc63-2d58-4cca-bfbd-e5d0a0adae9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:53.002804 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:53.002769 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod666e8e25_9dde_4db2_a3e2_3e75b5e24059.slice/crio-b9ef31d1db66056ef4713d8bb795f035d195117c07989b44d1198dc33baad823 WatchSource:0}: Error finding container b9ef31d1db66056ef4713d8bb795f035d195117c07989b44d1198dc33baad823: Status 404 returned error can't find the container with id b9ef31d1db66056ef4713d8bb795f035d195117c07989b44d1198dc33baad823 Jun 10 12:36:53.003979 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:53.003957 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9303f166_88cd_4695_8a92_ead714e32ca1.slice/crio-bc63c5291772d09cdaa59541381f3e61112408e7c024b2abea9e17ace09ac079 WatchSource:0}: Error finding container bc63c5291772d09cdaa59541381f3e61112408e7c024b2abea9e17ace09ac079: Status 404 returned error can't find the container with id bc63c5291772d09cdaa59541381f3e61112408e7c024b2abea9e17ace09ac079 Jun 10 12:36:53.005543 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:53.005510 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9f4f53_a664_4c7d_a457_9e0779ad3c16.slice/crio-06c76ee94708709f387051896ed2021bb566c9fbf7ee77839fd89625441d019f WatchSource:0}: Error finding container 06c76ee94708709f387051896ed2021bb566c9fbf7ee77839fd89625441d019f: Status 404 returned error can't find the container with id 06c76ee94708709f387051896ed2021bb566c9fbf7ee77839fd89625441d019f Jun 10 12:36:53.009245 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:53.009224 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd022e905_ba12_4217_8018_186dd3865aa5.slice/crio-8204c9a1bcbdd8f76a14e5d0c7246322a45bedb0b879250e6eb5784dbe788e64 WatchSource:0}: Error finding container 8204c9a1bcbdd8f76a14e5d0c7246322a45bedb0b879250e6eb5784dbe788e64: Status 404 returned error can't find the container with id 8204c9a1bcbdd8f76a14e5d0c7246322a45bedb0b879250e6eb5784dbe788e64 Jun 10 12:36:53.009792 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:53.009770 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa02a27_aa46_4bb2_b823_a91092066a54.slice/crio-aa5fda348eb3e939388c37df47bee15f2d153ce70316cdc0173fa1be0b60103b WatchSource:0}: Error finding container aa5fda348eb3e939388c37df47bee15f2d153ce70316cdc0173fa1be0b60103b: Status 404 returned error can't find the container with id aa5fda348eb3e939388c37df47bee15f2d153ce70316cdc0173fa1be0b60103b Jun 10 12:36:53.011276 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:53.011182 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd979c87_5eb1_4cef_a302_4ab9570eb4cd.slice/crio-69b89ec45b5485bae28bf65290cbd94a671ddc8fbd793fbeae211bc372b61440 WatchSource:0}: Error finding container 69b89ec45b5485bae28bf65290cbd94a671ddc8fbd793fbeae211bc372b61440: Status 404 returned error can't find the container with id 69b89ec45b5485bae28bf65290cbd94a671ddc8fbd793fbeae211bc372b61440 Jun 10 12:36:53.013178 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:53.013156 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3dc5bb_5a81_4b35_8a0c_7c9889737bef.slice/crio-db2302f50e1d46fdf9aabe2117b8b609b687f731c142fbdae58aba9147c15123 WatchSource:0}: Error finding container db2302f50e1d46fdf9aabe2117b8b609b687f731c142fbdae58aba9147c15123: Status 404 returned error can't find the container with id db2302f50e1d46fdf9aabe2117b8b609b687f731c142fbdae58aba9147c15123 Jun 10 12:36:53.014733 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:53.014043 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee00a67b_e3d6_4c47_81dc_d5a523e04dc7.slice/crio-aaa0e57c9807dfb5e915ac29de93cfc4f0e161a47a545b9f55de729534e0f770 WatchSource:0}: Error finding container aaa0e57c9807dfb5e915ac29de93cfc4f0e161a47a545b9f55de729534e0f770: Status 404 returned error can't find the container with id aaa0e57c9807dfb5e915ac29de93cfc4f0e161a47a545b9f55de729534e0f770 Jun 10 12:36:53.016370 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:36:53.014847 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95dae07_0e6a_4996_b135_98cff030bfa8.slice/crio-cf433110a535acc2f75f17f35bd92f01ed45832d2f988eea7bf53c1e2ae4a63f WatchSource:0}: Error finding container cf433110a535acc2f75f17f35bd92f01ed45832d2f988eea7bf53c1e2ae4a63f: Status 404 returned error can't find the container with id cf433110a535acc2f75f17f35bd92f01ed45832d2f988eea7bf53c1e2ae4a63f Jun 10 12:36:53.249050 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.249024 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" event={"ID":"c95dae07-0e6a-4996-b135-98cff030bfa8","Type":"ContainerStarted","Data":"cf433110a535acc2f75f17f35bd92f01ed45832d2f988eea7bf53c1e2ae4a63f"} Jun 10 12:36:53.250191 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.250169 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerStarted","Data":"aaa0e57c9807dfb5e915ac29de93cfc4f0e161a47a545b9f55de729534e0f770"} Jun 10 12:36:53.251377 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.251355 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jzvbs" event={"ID":"bd979c87-5eb1-4cef-a302-4ab9570eb4cd","Type":"ContainerStarted","Data":"69b89ec45b5485bae28bf65290cbd94a671ddc8fbd793fbeae211bc372b61440"} Jun 10 12:36:53.251463 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.251408 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 12:31:52 +0000 UTC" deadline="2028-02-10 20:59:06.62894298 +0000 UTC" Jun 10 12:36:53.251463 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.251437 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14648h22m13.377510215s" Jun 10 12:36:53.252478 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.252451 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" event={"ID":"8aa02a27-aa46-4bb2-b823-a91092066a54","Type":"ContainerStarted","Data":"aa5fda348eb3e939388c37df47bee15f2d153ce70316cdc0173fa1be0b60103b"} Jun 10 12:36:53.253617 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.253597 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nff49" event={"ID":"d022e905-ba12-4217-8018-186dd3865aa5","Type":"ContainerStarted","Data":"8204c9a1bcbdd8f76a14e5d0c7246322a45bedb0b879250e6eb5784dbe788e64"} Jun 10 12:36:53.254765 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.254747 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gqpm5" event={"ID":"af9f4f53-a664-4c7d-a457-9e0779ad3c16","Type":"ContainerStarted","Data":"06c76ee94708709f387051896ed2021bb566c9fbf7ee77839fd89625441d019f"} Jun 10 12:36:53.256046 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.256024 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-99zc2" event={"ID":"9303f166-88cd-4695-8a92-ead714e32ca1","Type":"ContainerStarted","Data":"bc63c5291772d09cdaa59541381f3e61112408e7c024b2abea9e17ace09ac079"} Jun 10 12:36:53.259132 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.259088 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" event={"ID":"89d633427a813ac026468ec864f4e82d","Type":"ContainerStarted","Data":"e1c248ef8b70894cb78ce302d4e9bf76f438c3f665870363d299192ade8e20a7"} Jun 10 12:36:53.260515 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.260496 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" event={"ID":"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef","Type":"ContainerStarted","Data":"db2302f50e1d46fdf9aabe2117b8b609b687f731c142fbdae58aba9147c15123"} Jun 10 12:36:53.261520 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.261502 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5m4pd" event={"ID":"666e8e25-9dde-4db2-a3e2-3e75b5e24059","Type":"ContainerStarted","Data":"b9ef31d1db66056ef4713d8bb795f035d195117c07989b44d1198dc33baad823"} Jun 10 12:36:53.281865 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.281805 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-99.ec2.internal" podStartSLOduration=2.281788113 podStartE2EDuration="2.281788113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:36:53.281445846 +0000 UTC m=+3.492291832" watchObservedRunningTime="2026-06-10 12:36:53.281788113 +0000 UTC m=+3.492634100" Jun 10 12:36:53.800334 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.800294 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:53.800503 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:53.800468 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:53.800575 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:53.800534 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret podName:63acd877-6555-4895-8e27-9b7540f95de8 nodeName:}" failed. No retries permitted until 2026-06-10 12:36:55.800516486 +0000 UTC m=+6.011362455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret") pod "global-pull-secret-syncer-fww7p" (UID: "63acd877-6555-4895-8e27-9b7540f95de8") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:53.900704 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.900658 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:53.900886 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:53.900709 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:53.900950 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:53.900884 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:36:53.900950 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:53.900902 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:36:53.900950 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:53.900915 2582 projected.go:196] Error preparing data for projected volume kube-api-access-52fr9 for pod openshift-network-diagnostics/network-check-target-pf78n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:53.901131 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:53.900974 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9 podName:491acc63-2d58-4cca-bfbd-e5d0a0adae9a nodeName:}" failed. No retries permitted until 2026-06-10 12:36:55.900957003 +0000 UTC m=+6.111802975 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-52fr9" (UniqueName: "kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9") pod "network-check-target-pf78n" (UID: "491acc63-2d58-4cca-bfbd-e5d0a0adae9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:53.901463 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:53.901444 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:36:53.901529 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:53.901504 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:36:55.901488049 +0000 UTC m=+6.112334017 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:36:54.243275 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:54.243239 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:54.243463 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:54.243373 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:36:54.243463 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:54.243239 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:54.243591 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:54.243476 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:36:54.243591 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:54.243525 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:54.243684 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:54.243592 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:36:54.252710 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:54.252617 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 12:31:52 +0000 UTC" deadline="2028-02-08 07:00:41.001672337 +0000 UTC" Jun 10 12:36:54.252710 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:54.252671 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14586h23m46.749006135s" Jun 10 12:36:54.276032 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:54.275993 2582 generic.go:356] "Generic (PLEG): container finished" podID="cdfca2d07c4ff59b8cccc8704407fc86" containerID="cba12c975d715b2d5c6a61d90f21431ddd357864920110d1aa6e8cc88a3a929a" exitCode=0 Jun 10 12:36:54.276223 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:54.276080 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" event={"ID":"cdfca2d07c4ff59b8cccc8704407fc86","Type":"ContainerDied","Data":"cba12c975d715b2d5c6a61d90f21431ddd357864920110d1aa6e8cc88a3a929a"} Jun 10 12:36:55.303630 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:55.302945 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" event={"ID":"cdfca2d07c4ff59b8cccc8704407fc86","Type":"ContainerStarted","Data":"ae004eca1ad21071d947026e4a220b2495d53705e09b2564212808bb9d4f0fa0"} Jun 10 12:36:55.818344 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:55.817822 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:55.818344 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:55.817956 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:55.818344 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:55.818014 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret podName:63acd877-6555-4895-8e27-9b7540f95de8 nodeName:}" failed. No retries permitted until 2026-06-10 12:36:59.817997391 +0000 UTC m=+10.028843360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret") pod "global-pull-secret-syncer-fww7p" (UID: "63acd877-6555-4895-8e27-9b7540f95de8") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:55.919068 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:55.918233 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:55.919068 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:55.918282 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:55.919068 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:55.918431 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:36:55.919068 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:55.918451 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:36:55.919068 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:55.918463 2582 projected.go:196] Error preparing data for projected volume kube-api-access-52fr9 for pod openshift-network-diagnostics/network-check-target-pf78n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:55.919068 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:55.918523 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9 podName:491acc63-2d58-4cca-bfbd-e5d0a0adae9a nodeName:}" failed. No retries permitted until 2026-06-10 12:36:59.918505523 +0000 UTC m=+10.129351506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-52fr9" (UniqueName: "kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9") pod "network-check-target-pf78n" (UID: "491acc63-2d58-4cca-bfbd-e5d0a0adae9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:55.919068 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:55.918925 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:36:55.919068 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:55.918976 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:36:59.918961542 +0000 UTC m=+10.129807512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:36:56.242398 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:56.241857 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:56.242398 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:56.241857 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:56.242398 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:56.241986 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:36:56.242398 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:56.242003 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:56.242398 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:56.242139 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:36:56.242398 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:56.242233 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:36:58.243232 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:58.243185 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:58.243772 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:58.243234 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:58.243772 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:58.243186 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:58.243772 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:58.243334 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:36:58.243772 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:58.243445 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:36:58.243772 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:58.243523 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:36:59.851823 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:59.851781 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:36:59.852438 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:59.851942 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:59.852438 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:59.852004 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret podName:63acd877-6555-4895-8e27-9b7540f95de8 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:07.851985949 +0000 UTC m=+18.062831937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret") pod "global-pull-secret-syncer-fww7p" (UID: "63acd877-6555-4895-8e27-9b7540f95de8") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:36:59.952904 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:59.952475 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:36:59.952904 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:36:59.952523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:36:59.952904 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:59.952664 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:36:59.952904 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:59.952681 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:36:59.952904 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:59.952694 2582 projected.go:196] Error preparing data for projected volume kube-api-access-52fr9 for pod openshift-network-diagnostics/network-check-target-pf78n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:59.952904 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:59.952745 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:36:59.952904 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:59.952754 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9 podName:491acc63-2d58-4cca-bfbd-e5d0a0adae9a nodeName:}" failed. No retries permitted until 2026-06-10 12:37:07.952737522 +0000 UTC m=+18.163583500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-52fr9" (UniqueName: "kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9") pod "network-check-target-pf78n" (UID: "491acc63-2d58-4cca-bfbd-e5d0a0adae9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:36:59.952904 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:36:59.952819 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:07.952804635 +0000 UTC m=+18.163650602 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:37:00.246634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:00.245987 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:00.246634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:00.246061 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:00.246634 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:00.246120 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:00.246634 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:00.246364 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:00.246634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:00.246453 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:00.246634 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:00.246553 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:02.241924 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:02.241731 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:02.242415 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:02.241776 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:02.242415 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:02.242028 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:02.242415 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:02.241878 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:02.242415 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:02.242138 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:02.242415 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:02.242193 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:04.242081 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:04.242042 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:04.242543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:04.242043 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:04.242543 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:04.242189 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:04.242543 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:04.242291 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:04.242543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:04.242307 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:04.242543 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:04.242379 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:06.242278 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:06.242245 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:06.242773 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:06.242244 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:06.242773 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:06.242399 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:06.242773 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:06.242253 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:06.242773 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:06.242456 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:06.242773 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:06.242532 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:07.910215 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:07.910173 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:07.910654 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:07.910315 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:37:07.910654 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:07.910391 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret podName:63acd877-6555-4895-8e27-9b7540f95de8 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:23.910371285 +0000 UTC m=+34.121217270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret") pod "global-pull-secret-syncer-fww7p" (UID: "63acd877-6555-4895-8e27-9b7540f95de8") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:37:08.010791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:08.010751 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:08.010791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:08.010797 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:08.011030 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:08.010913 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:37:08.011030 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:08.010912 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:37:08.011030 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:08.010936 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:37:08.011030 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:08.010951 2582 projected.go:196] Error preparing data for projected volume kube-api-access-52fr9 for pod openshift-network-diagnostics/network-check-target-pf78n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:37:08.011030 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:08.011003 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:24.010981394 +0000 UTC m=+34.221827382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:37:08.011030 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:08.011026 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9 podName:491acc63-2d58-4cca-bfbd-e5d0a0adae9a nodeName:}" failed. No retries permitted until 2026-06-10 12:37:24.011014549 +0000 UTC m=+34.221860513 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-52fr9" (UniqueName: "kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9") pod "network-check-target-pf78n" (UID: "491acc63-2d58-4cca-bfbd-e5d0a0adae9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:37:08.242311 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:08.242218 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:08.242475 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:08.242218 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:08.242475 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:08.242383 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:08.242475 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:08.242397 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:08.242630 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:08.242513 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:08.242630 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:08.242594 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:10.242471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:10.242442 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:10.242947 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:10.242553 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:10.242947 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:10.242641 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:10.242947 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:10.242749 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:10.242947 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:10.242795 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:10.242947 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:10.242872 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:12.241738 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.241567 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:12.241738 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.241591 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:12.241738 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:12.241703 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:12.241738 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.241567 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:12.242467 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:12.241798 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:12.242467 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:12.241875 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:12.339833 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.339802 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" event={"ID":"c95dae07-0e6a-4996-b135-98cff030bfa8","Type":"ContainerStarted","Data":"c1dad811cab23d8c7730c1f2c4b9e4b0c5086e7f246d21e18a3884c3e61e7287"} Jun 10 12:37:12.342089 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.342070 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:37:12.342455 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.342431 2582 generic.go:356] "Generic (PLEG): container finished" podID="ee00a67b-e3d6-4c47-81dc-d5a523e04dc7" containerID="e0179b78e42ec5a871f737076dc54628218eefd6f17137a0d4f4d0c5599bd71b" exitCode=1 Jun 10 12:37:12.342528 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.342501 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerDied","Data":"e0179b78e42ec5a871f737076dc54628218eefd6f17137a0d4f4d0c5599bd71b"} Jun 10 12:37:12.342607 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.342589 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerStarted","Data":"b689e7b99e29a514d2fb55d3b945284751d652dfc4fe68b9e32acf2a1d62b0c6"} Jun 10 12:37:12.344321 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.344283 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jzvbs" event={"ID":"bd979c87-5eb1-4cef-a302-4ab9570eb4cd","Type":"ContainerStarted","Data":"fcb8a420fc6fca84146211f4d286118702b845716b7a17166edca8f9036b8e8d"} Jun 10 12:37:12.345970 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.345949 2582 generic.go:356] "Generic (PLEG): container finished" podID="8aa02a27-aa46-4bb2-b823-a91092066a54" containerID="42a7a47a91f13704b4e3f4e1a849642b2d93e0fb970358734985f71c5ed258fe" exitCode=0 Jun 10 12:37:12.346063 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.346037 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" event={"ID":"8aa02a27-aa46-4bb2-b823-a91092066a54","Type":"ContainerDied","Data":"42a7a47a91f13704b4e3f4e1a849642b2d93e0fb970358734985f71c5ed258fe"} Jun 10 12:37:12.348133 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.347991 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gqpm5" event={"ID":"af9f4f53-a664-4c7d-a457-9e0779ad3c16","Type":"ContainerStarted","Data":"d9cec5a8a205d3df870c32bfde609b8fc22d93266900c9f85497993efd0f8d33"} Jun 10 12:37:12.349628 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.349607 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-99zc2" event={"ID":"9303f166-88cd-4695-8a92-ead714e32ca1","Type":"ContainerStarted","Data":"d15b1bec1575132bb6cee2ad70986407e66047eb4edfc2575ddfe97e0d2f0df7"} Jun 10 12:37:12.351195 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.351173 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" event={"ID":"fb3dc5bb-5a81-4b35-8a0c-7c9889737bef","Type":"ContainerStarted","Data":"f80fef2b448c65b31b4bb883f092dd9e064e25bdfc6ba22910bd1a4fbf15f977"} Jun 10 12:37:12.352506 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.352481 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5m4pd" event={"ID":"666e8e25-9dde-4db2-a3e2-3e75b5e24059","Type":"ContainerStarted","Data":"fa40c6d7274d99467aa699736fbda446175a073de91e38aa16b4f58246012697"} Jun 10 12:37:12.367379 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.367334 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-99.ec2.internal" podStartSLOduration=21.367320726 podStartE2EDuration="21.367320726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:36:55.32476523 +0000 UTC m=+5.535611218" watchObservedRunningTime="2026-06-10 12:37:12.367320726 +0000 UTC m=+22.578166714" Jun 10 12:37:12.390014 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.389975 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jzvbs" podStartSLOduration=3.471556175 podStartE2EDuration="22.389959001s" totalImagesPullingTime="18.918402826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:12.366898729 +0000 UTC m=+22.577744718" watchObservedRunningTime="2026-06-10 12:37:12.389959001 +0000 UTC m=+22.600804979" Jun 10 12:37:12.390508 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.390485 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gqpm5" podStartSLOduration=3.537039365 podStartE2EDuration="22.390480354s" totalImagesPullingTime="18.853440989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:12.389177633 +0000 UTC m=+22.600023617" watchObservedRunningTime="2026-06-10 12:37:12.390480354 +0000 UTC m=+22.601326368" Jun 10 12:37:12.461519 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.461474 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8s6gl" podStartSLOduration=3.592613989 podStartE2EDuration="22.461458709s" totalImagesPullingTime="18.86884472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:12.461328841 +0000 UTC m=+22.672174828" watchObservedRunningTime="2026-06-10 12:37:12.461458709 +0000 UTC m=+22.672304697" Jun 10 12:37:12.462050 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.462013 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-99zc2" podStartSLOduration=3.606722318 podStartE2EDuration="22.462006334s" totalImagesPullingTime="18.855284016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:12.418488116 +0000 UTC m=+22.629334102" watchObservedRunningTime="2026-06-10 12:37:12.462006334 +0000 UTC m=+22.672852375" Jun 10 12:37:12.482894 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:12.482853 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5m4pd" podStartSLOduration=3.5902091130000002 podStartE2EDuration="22.482837085s" totalImagesPullingTime="18.892627972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:12.482490031 +0000 UTC m=+22.693336019" watchObservedRunningTime="2026-06-10 12:37:12.482837085 +0000 UTC m=+22.693683072" Jun 10 12:37:13.357541 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:13.357338 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:37:13.358148 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:13.358120 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerStarted","Data":"4c05e84bd1b2d395be5540c8810d5e9435fa3e535fe0e111f5d22bbfb35c35ef"} Jun 10 12:37:13.358244 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:13.358232 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerStarted","Data":"fd3aedd2bc155f1dec9b4e9ee41e53b35771e0b3b7d28ee9b9d7fe28b566235c"} Jun 10 12:37:13.358284 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:13.358269 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerStarted","Data":"57b85805fb231fa5a782ed672749712a0c86ff94462b331a9f9524cf7ded850b"} Jun 10 12:37:13.358323 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:13.358299 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerStarted","Data":"f138dd8104b717294410ecee52707efb791c6df98a1ea45df1aa1bf5151cb3ad"} Jun 10 12:37:13.359682 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:13.359658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nff49" event={"ID":"d022e905-ba12-4217-8018-186dd3865aa5","Type":"ContainerStarted","Data":"3ee7267976ddda6539e528ec98a9bcfcc47f7ef53400e3fda40a9236caa6778f"} Jun 10 12:37:13.388824 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:13.388380 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nff49" podStartSLOduration=4.538145542 podStartE2EDuration="23.388359704s" totalImagesPullingTime="18.850214162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:13.38825813 +0000 UTC m=+23.599104118" watchObservedRunningTime="2026-06-10 12:37:13.388359704 +0000 UTC m=+23.599205692" Jun 10 12:37:13.398857 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:13.398824 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 10 12:37:14.242527 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:14.242429 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:14.242527 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:14.242433 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:14.242763 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:14.242564 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:14.242763 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:14.242471 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-10T12:37:13.398841597Z","UUID":"2670c6c7-5e80-40a5-830e-756326ea7014","Handler":null,"Name":"","Endpoint":""} Jun 10 12:37:14.242763 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:14.242640 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:14.242763 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:14.242433 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:14.242763 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:14.242737 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:14.244198 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:14.244175 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 10 12:37:14.244327 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:14.244210 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 10 12:37:14.364497 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:14.364394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" event={"ID":"c95dae07-0e6a-4996-b135-98cff030bfa8","Type":"ContainerStarted","Data":"b766e9e46d17bf0b542eaf9e6fae155e8660ae5ccafef8e198386f9fb890d59f"} Jun 10 12:37:15.368601 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:15.368561 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" event={"ID":"c95dae07-0e6a-4996-b135-98cff030bfa8","Type":"ContainerStarted","Data":"fc3fe5f79eb6af28df9657947ce84daf5f82708c273df4a958ee85dc255d857b"} Jun 10 12:37:16.016600 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:16.016564 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:37:16.017258 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:16.017236 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:37:16.049815 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:16.049761 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x66km" podStartSLOduration=5.195573809 podStartE2EDuration="26.04974396s" totalImagesPullingTime="20.854170151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:15.408978645 +0000 UTC m=+25.619824632" watchObservedRunningTime="2026-06-10 12:37:16.04974396 +0000 UTC m=+26.260590009" Jun 10 12:37:16.244841 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:16.244809 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:16.244983 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:16.244898 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:16.244983 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:16.244912 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:16.244983 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:16.244923 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:16.245129 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:16.244980 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:16.245129 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:16.245041 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:16.370441 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:16.370409 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:37:16.371007 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:16.370772 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-99zc2" Jun 10 12:37:17.374568 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:17.374543 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:37:17.374966 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:17.374880 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerStarted","Data":"fac2a0eeabb0d6a8a09d2af82a75d88a5f03a92149554003e457c95288ebc59b"} Jun 10 12:37:17.376498 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:17.376474 2582 generic.go:356] "Generic (PLEG): container finished" podID="8aa02a27-aa46-4bb2-b823-a91092066a54" containerID="e826888fe0bf034b28875f877fe1dccf9a41b149a772ae423a220c996b3607ef" exitCode=0 Jun 10 12:37:17.376606 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:17.376554 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" event={"ID":"8aa02a27-aa46-4bb2-b823-a91092066a54","Type":"ContainerDied","Data":"e826888fe0bf034b28875f877fe1dccf9a41b149a772ae423a220c996b3607ef"} Jun 10 12:37:18.242359 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:18.242131 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:18.242359 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:18.242132 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:18.242573 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:18.242172 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:18.242573 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:18.242420 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:18.242573 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:18.242542 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:18.242712 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:18.242639 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:18.380873 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:18.380843 2582 generic.go:356] "Generic (PLEG): container finished" podID="8aa02a27-aa46-4bb2-b823-a91092066a54" containerID="ac63fe2f2a7d2147ee9392a6599c767ffa7d60c15c3933fe1a09c13e0db52343" exitCode=0 Jun 10 12:37:18.381236 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:18.380934 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" event={"ID":"8aa02a27-aa46-4bb2-b823-a91092066a54","Type":"ContainerDied","Data":"ac63fe2f2a7d2147ee9392a6599c767ffa7d60c15c3933fe1a09c13e0db52343"} Jun 10 12:37:19.386162 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.386131 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:37:19.386579 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.386551 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerStarted","Data":"e2c2b830103c3ba5913fb9780678baa16b44a21526e33d834124e25c8e8d8751"} Jun 10 12:37:19.386824 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.386796 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:37:19.386962 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.386859 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:37:19.386962 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.386901 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:37:19.387067 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.386986 2582 scope.go:117] "RemoveContainer" containerID="e0179b78e42ec5a871f737076dc54628218eefd6f17137a0d4f4d0c5599bd71b" Jun 10 12:37:19.389354 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.389322 2582 generic.go:356] "Generic (PLEG): container finished" podID="8aa02a27-aa46-4bb2-b823-a91092066a54" containerID="f89c2578e6b9ee6a1297ca5b0360b932f5d65f8d1c4e8d74450a12523599edaf" exitCode=0 Jun 10 12:37:19.389442 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.389360 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" event={"ID":"8aa02a27-aa46-4bb2-b823-a91092066a54","Type":"ContainerDied","Data":"f89c2578e6b9ee6a1297ca5b0360b932f5d65f8d1c4e8d74450a12523599edaf"} Jun 10 12:37:19.404419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.404397 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:37:19.404729 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:19.404711 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:37:20.242607 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.242534 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:20.242795 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:20.242637 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:20.242795 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.242714 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:20.242795 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.242774 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:20.242960 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:20.242873 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:20.243020 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:20.242977 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:20.326704 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.326674 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:20.396366 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.396278 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:37:20.396923 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.396690 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" event={"ID":"ee00a67b-e3d6-4c47-81dc-d5a523e04dc7","Type":"ContainerStarted","Data":"9b1e17a693cf0113353c7425e488dc38a8090caf4728a7ef42c205d178d85ace"} Jun 10 12:37:20.416373 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.416290 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:20.434910 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.434851 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" podStartSLOduration=11.491021811 podStartE2EDuration="30.434835237s" totalImagesPullingTime="18.943813426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:20.434405775 +0000 UTC m=+30.645251764" watchObservedRunningTime="2026-06-10 12:37:20.434835237 +0000 UTC m=+30.645681223" Jun 10 12:37:20.515984 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.515916 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fww7p"] Jun 10 12:37:20.516141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.516013 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:20.516213 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:20.516171 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:20.519136 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.519110 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zlgzd"] Jun 10 12:37:20.519259 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.519220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:20.519356 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:20.519331 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:20.541189 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.541156 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pf78n"] Jun 10 12:37:20.541329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.541291 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:20.541460 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:20.541394 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:20.862546 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.862515 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:20.961834 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:20.961786 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:22.103877 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:22.103843 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:22.170789 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:22.170750 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:22.241910 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:22.241875 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:22.242073 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:22.241874 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:22.242073 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:22.242012 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:22.242216 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:22.242136 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:22.242216 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:22.242154 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:22.242312 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:22.242241 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:23.923795 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:23.923770 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:23.924122 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:23.923906 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:37:23.924122 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:23.923963 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret podName:63acd877-6555-4895-8e27-9b7540f95de8 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:55.923946171 +0000 UTC m=+66.134792137 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret") pod "global-pull-secret-syncer-fww7p" (UID: "63acd877-6555-4895-8e27-9b7540f95de8") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:37:24.024551 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.024523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:24.024689 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.024556 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:24.024689 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.024674 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:37:24.024802 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.024739 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:56.024720579 +0000 UTC m=+66.235566549 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:37:24.024802 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.024676 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:37:24.024802 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.024772 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:37:24.024802 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.024781 2582 projected.go:196] Error preparing data for projected volume kube-api-access-52fr9 for pod openshift-network-diagnostics/network-check-target-pf78n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:37:24.024998 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.024821 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9 podName:491acc63-2d58-4cca-bfbd-e5d0a0adae9a nodeName:}" failed. No retries permitted until 2026-06-10 12:37:56.024809438 +0000 UTC m=+66.235655404 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-52fr9" (UniqueName: "kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9") pod "network-check-target-pf78n" (UID: "491acc63-2d58-4cca-bfbd-e5d0a0adae9a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:37:24.242150 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.242059 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:24.242290 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.242188 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:24.242290 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.242192 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fww7p" podUID="63acd877-6555-4895-8e27-9b7540f95de8" Jun 10 12:37:24.242290 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.242220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:24.242400 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.242315 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:37:24.242434 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.242400 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pf78n" podUID="491acc63-2d58-4cca-bfbd-e5d0a0adae9a" Jun 10 12:37:24.410490 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.410454 2582 generic.go:356] "Generic (PLEG): container finished" podID="8aa02a27-aa46-4bb2-b823-a91092066a54" containerID="f669e1efbf33c637769bd1360f39c7675cae7365c60489c69d6f5c5a87b62c64" exitCode=0 Jun 10 12:37:24.410490 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.410493 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" event={"ID":"8aa02a27-aa46-4bb2-b823-a91092066a54","Type":"ContainerDied","Data":"f669e1efbf33c637769bd1360f39c7675cae7365c60489c69d6f5c5a87b62c64"} Jun 10 12:37:24.631986 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.631953 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-99.ec2.internal" event="NodeReady" Jun 10 12:37:24.632126 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.632063 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 10 12:37:24.684437 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.684410 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vmlx9"] Jun 10 12:37:24.693996 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.693979 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f5d7t"] Jun 10 12:37:24.694141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.694125 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.697056 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.696883 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 10 12:37:24.697163 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.696962 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 10 12:37:24.697163 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.696962 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hpqfd\"" Jun 10 12:37:24.704690 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.704672 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f5d7t"] Jun 10 12:37:24.704814 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.704761 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f5d7t" Jun 10 12:37:24.704865 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.704814 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vmlx9"] Jun 10 12:37:24.707464 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.707445 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 10 12:37:24.707570 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.707513 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 10 12:37:24.707640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.707604 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9kr2v\"" Jun 10 12:37:24.707801 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.707782 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 10 12:37:24.829768 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.829738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95d21008-75a7-4b76-8a26-ede242d95fd4-cert\") pod \"ingress-canary-f5d7t\" (UID: \"95d21008-75a7-4b76-8a26-ede242d95fd4\") " pod="openshift-ingress-canary/ingress-canary-f5d7t" Jun 10 12:37:24.829957 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.829859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306d7497-fa10-42e3-8c0a-7fb50bf91206-tmp-dir\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.829957 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.829889 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.829957 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.829917 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jtdd\" (UniqueName: \"kubernetes.io/projected/95d21008-75a7-4b76-8a26-ede242d95fd4-kube-api-access-8jtdd\") pod \"ingress-canary-f5d7t\" (UID: \"95d21008-75a7-4b76-8a26-ede242d95fd4\") " pod="openshift-ingress-canary/ingress-canary-f5d7t" Jun 10 12:37:24.830091 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.830026 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mndx\" (UniqueName: \"kubernetes.io/projected/306d7497-fa10-42e3-8c0a-7fb50bf91206-kube-api-access-9mndx\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.830091 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.830056 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/306d7497-fa10-42e3-8c0a-7fb50bf91206-config-volume\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.930984 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.930954 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306d7497-fa10-42e3-8c0a-7fb50bf91206-tmp-dir\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.931419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.930990 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.931419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.931011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jtdd\" (UniqueName: \"kubernetes.io/projected/95d21008-75a7-4b76-8a26-ede242d95fd4-kube-api-access-8jtdd\") pod \"ingress-canary-f5d7t\" (UID: \"95d21008-75a7-4b76-8a26-ede242d95fd4\") " pod="openshift-ingress-canary/ingress-canary-f5d7t" Jun 10 12:37:24.931419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.931046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mndx\" (UniqueName: \"kubernetes.io/projected/306d7497-fa10-42e3-8c0a-7fb50bf91206-kube-api-access-9mndx\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.931419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.931075 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/306d7497-fa10-42e3-8c0a-7fb50bf91206-config-volume\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.931419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.931128 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95d21008-75a7-4b76-8a26-ede242d95fd4-cert\") pod \"ingress-canary-f5d7t\" (UID: \"95d21008-75a7-4b76-8a26-ede242d95fd4\") " pod="openshift-ingress-canary/ingress-canary-f5d7t" Jun 10 12:37:24.931419 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.931134 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:37:24.931419 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:24.931202 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls podName:306d7497-fa10-42e3-8c0a-7fb50bf91206 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:25.431182077 +0000 UTC m=+35.642028043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls") pod "dns-default-vmlx9" (UID: "306d7497-fa10-42e3-8c0a-7fb50bf91206") : secret "dns-default-metrics-tls" not found Jun 10 12:37:24.931419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.931351 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306d7497-fa10-42e3-8c0a-7fb50bf91206-tmp-dir\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.935293 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.935272 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95d21008-75a7-4b76-8a26-ede242d95fd4-cert\") pod \"ingress-canary-f5d7t\" (UID: \"95d21008-75a7-4b76-8a26-ede242d95fd4\") " pod="openshift-ingress-canary/ingress-canary-f5d7t" Jun 10 12:37:24.939855 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.939835 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jtdd\" (UniqueName: \"kubernetes.io/projected/95d21008-75a7-4b76-8a26-ede242d95fd4-kube-api-access-8jtdd\") pod \"ingress-canary-f5d7t\" (UID: \"95d21008-75a7-4b76-8a26-ede242d95fd4\") " pod="openshift-ingress-canary/ingress-canary-f5d7t" Jun 10 12:37:24.940808 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.940787 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/306d7497-fa10-42e3-8c0a-7fb50bf91206-config-volume\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:24.942440 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:24.942417 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mndx\" (UniqueName: \"kubernetes.io/projected/306d7497-fa10-42e3-8c0a-7fb50bf91206-kube-api-access-9mndx\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:25.013673 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:25.013638 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f5d7t" Jun 10 12:37:25.201686 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:25.201655 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f5d7t"] Jun 10 12:37:25.205066 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:37:25.205038 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d21008_75a7_4b76_8a26_ede242d95fd4.slice/crio-45b014decd7b321b30ba14d9dc82b619dd1d42406f8fd567d7958d2549e7f6c2 WatchSource:0}: Error finding container 45b014decd7b321b30ba14d9dc82b619dd1d42406f8fd567d7958d2549e7f6c2: Status 404 returned error can't find the container with id 45b014decd7b321b30ba14d9dc82b619dd1d42406f8fd567d7958d2549e7f6c2 Jun 10 12:37:25.414072 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:25.413985 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f5d7t" event={"ID":"95d21008-75a7-4b76-8a26-ede242d95fd4","Type":"ContainerStarted","Data":"45b014decd7b321b30ba14d9dc82b619dd1d42406f8fd567d7958d2549e7f6c2"} Jun 10 12:37:25.416718 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:25.416693 2582 generic.go:356] "Generic (PLEG): container finished" podID="8aa02a27-aa46-4bb2-b823-a91092066a54" containerID="91031a0f552467c52aff88234caafcec8e05ad36405f8d5dca61ba54c0caa7b5" exitCode=0 Jun 10 12:37:25.416832 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:25.416739 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" event={"ID":"8aa02a27-aa46-4bb2-b823-a91092066a54","Type":"ContainerDied","Data":"91031a0f552467c52aff88234caafcec8e05ad36405f8d5dca61ba54c0caa7b5"} Jun 10 12:37:25.434495 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:25.434469 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:25.434586 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:25.434574 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:37:25.434633 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:25.434623 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls podName:306d7497-fa10-42e3-8c0a-7fb50bf91206 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:26.434611412 +0000 UTC m=+36.645457378 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls") pod "dns-default-vmlx9" (UID: "306d7497-fa10-42e3-8c0a-7fb50bf91206") : secret "dns-default-metrics-tls" not found Jun 10 12:37:25.739402 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:25.739314 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:25.799384 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:25.799347 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:26.241519 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.241484 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:26.241973 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.241604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:26.241973 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.241484 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:26.244464 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.244436 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 12:37:26.245934 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.245777 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 12:37:26.245934 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.245784 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vpjkq\"" Jun 10 12:37:26.245934 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.245814 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 12:37:26.245934 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.245843 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 12:37:26.245934 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.245814 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmpcd\"" Jun 10 12:37:26.422201 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.421926 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" event={"ID":"8aa02a27-aa46-4bb2-b823-a91092066a54","Type":"ContainerStarted","Data":"ebd8f60be8a26b7972affc3151b9de784afd209f55c9acce22b497a87d7d43fa"} Jun 10 12:37:26.443480 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.443448 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:26.443619 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:26.443586 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:37:26.443681 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:26.443643 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls podName:306d7497-fa10-42e3-8c0a-7fb50bf91206 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:28.443627903 +0000 UTC m=+38.654473868 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls") pod "dns-default-vmlx9" (UID: "306d7497-fa10-42e3-8c0a-7fb50bf91206") : secret "dns-default-metrics-tls" not found Jun 10 12:37:26.449488 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.449440 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z7vkn" podStartSLOduration=7.775828188 podStartE2EDuration="36.449421366s" totalImagesPullingTime="28.673593178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:26.447247838 +0000 UTC m=+36.658093824" watchObservedRunningTime="2026-06-10 12:37:26.449421366 +0000 UTC m=+36.660267349" Jun 10 12:37:26.584414 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.584383 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:26.695452 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:26.695416 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:28.427795 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:28.427762 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f5d7t" event={"ID":"95d21008-75a7-4b76-8a26-ede242d95fd4","Type":"ContainerStarted","Data":"4a71f63de1be2b936ba4bd9d89558c97d38ae656924059a58866db2520c3e961"} Jun 10 12:37:28.450534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:28.450492 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f5d7t" podStartSLOduration=2.099596313 podStartE2EDuration="4.450477496s" totalImagesPullingTime="2.350881183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:37:28.448728514 +0000 UTC m=+38.659574502" watchObservedRunningTime="2026-06-10 12:37:28.450477496 +0000 UTC m=+38.661323460" Jun 10 12:37:28.457211 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:28.457189 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:28.457343 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:28.457303 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:37:28.457402 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:28.457362 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls podName:306d7497-fa10-42e3-8c0a-7fb50bf91206 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:32.457347255 +0000 UTC m=+42.668193233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls") pod "dns-default-vmlx9" (UID: "306d7497-fa10-42e3-8c0a-7fb50bf91206") : secret "dns-default-metrics-tls" not found Jun 10 12:37:29.941370 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:29.941336 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:30.000085 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:30.000055 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:30.390491 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:30.390452 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:30.489063 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:30.489027 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:32.242641 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:32.242603 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:32.307503 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:32.307470 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:32.481861 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:32.481822 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:32.482023 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:32.481934 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:37:32.482023 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:32.481987 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls podName:306d7497-fa10-42e3-8c0a-7fb50bf91206 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:40.481972022 +0000 UTC m=+50.692817988 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls") pod "dns-default-vmlx9" (UID: "306d7497-fa10-42e3-8c0a-7fb50bf91206") : secret "dns-default-metrics-tls" not found Jun 10 12:37:40.535528 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:40.535486 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:40.535993 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:40.535628 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:37:40.535993 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:40.535681 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls podName:306d7497-fa10-42e3-8c0a-7fb50bf91206 nodeName:}" failed. No retries permitted until 2026-06-10 12:37:56.535667523 +0000 UTC m=+66.746513487 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls") pod "dns-default-vmlx9" (UID: "306d7497-fa10-42e3-8c0a-7fb50bf91206") : secret "dns-default-metrics-tls" not found Jun 10 12:37:43.863884 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:43.863841 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:43.934770 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:43.934738 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:51.411435 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:51.411409 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tfsmb" Jun 10 12:37:51.462180 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:51.462147 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:51.678668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:51.678572 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:51.754017 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:51.753983 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:52.365143 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:52.365089 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:52.426109 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:52.426077 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:55.943025 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:55.942987 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:55.946153 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:55.946131 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 12:37:55.957486 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:55.957452 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/63acd877-6555-4895-8e27-9b7540f95de8-original-pull-secret\") pod \"global-pull-secret-syncer-fww7p\" (UID: \"63acd877-6555-4895-8e27-9b7540f95de8\") " pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:55.960337 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:55.960319 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fww7p" Jun 10 12:37:56.043526 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.043491 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:37:56.043657 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.043533 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:56.053009 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.052979 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 12:37:56.053191 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.053078 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 12:37:56.054455 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:56.054403 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 12:37:56.054614 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:56.054491 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:39:00.054465697 +0000 UTC m=+130.265311685 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : secret "metrics-daemon-secret" not found Jun 10 12:37:56.056795 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.056778 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 12:37:56.067826 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.067802 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52fr9\" (UniqueName: \"kubernetes.io/projected/491acc63-2d58-4cca-bfbd-e5d0a0adae9a-kube-api-access-52fr9\") pod \"network-check-target-pf78n\" (UID: \"491acc63-2d58-4cca-bfbd-e5d0a0adae9a\") " pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:56.088262 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.088233 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fww7p"] Jun 10 12:37:56.091211 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:37:56.091177 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63acd877_6555_4895_8e27_9b7540f95de8.slice/crio-6749db2a2b6820755a858f8d94df6fb53aa898fd77a84af448c5047fb159f4d8 WatchSource:0}: Error finding container 6749db2a2b6820755a858f8d94df6fb53aa898fd77a84af448c5047fb159f4d8: Status 404 returned error can't find the container with id 6749db2a2b6820755a858f8d94df6fb53aa898fd77a84af448c5047fb159f4d8 Jun 10 12:37:56.267630 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.267551 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmpcd\"" Jun 10 12:37:56.275878 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.275855 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:37:56.402800 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.402767 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pf78n"] Jun 10 12:37:56.405998 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:37:56.405971 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491acc63_2d58_4cca_bfbd_e5d0a0adae9a.slice/crio-0369c407151627779eb6bf09187878a145ff1a84204a7f76b37dc1ed49490689 WatchSource:0}: Error finding container 0369c407151627779eb6bf09187878a145ff1a84204a7f76b37dc1ed49490689: Status 404 returned error can't find the container with id 0369c407151627779eb6bf09187878a145ff1a84204a7f76b37dc1ed49490689 Jun 10 12:37:56.492348 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.492302 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fww7p" event={"ID":"63acd877-6555-4895-8e27-9b7540f95de8","Type":"ContainerStarted","Data":"6749db2a2b6820755a858f8d94df6fb53aa898fd77a84af448c5047fb159f4d8"} Jun 10 12:37:56.493543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.493511 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pf78n" event={"ID":"491acc63-2d58-4cca-bfbd-e5d0a0adae9a","Type":"ContainerStarted","Data":"0369c407151627779eb6bf09187878a145ff1a84204a7f76b37dc1ed49490689"} Jun 10 12:37:56.554449 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:56.554418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:37:56.554620 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:56.554579 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:37:56.554724 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:37:56.554649 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls podName:306d7497-fa10-42e3-8c0a-7fb50bf91206 nodeName:}" failed. No retries permitted until 2026-06-10 12:38:28.5546316 +0000 UTC m=+98.765477568 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls") pod "dns-default-vmlx9" (UID: "306d7497-fa10-42e3-8c0a-7fb50bf91206") : secret "dns-default-metrics-tls" not found Jun 10 12:37:57.705159 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:57.705121 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:37:57.772358 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:37:57.772312 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:38:01.507816 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:38:01.507776 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pf78n" event={"ID":"491acc63-2d58-4cca-bfbd-e5d0a0adae9a","Type":"ContainerStarted","Data":"8cde5466b260e0a73f291b3248ffae08bd90c67b4f50e64c1ae7fd8ae2959e6f"} Jun 10 12:38:01.508244 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:38:01.507870 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:38:01.509134 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:38:01.509092 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fww7p" event={"ID":"63acd877-6555-4895-8e27-9b7540f95de8","Type":"ContainerStarted","Data":"f687e159c3dfd69014198cfcdef75cfd0b97497b624b58713126b5c4bcaa068b"} Jun 10 12:38:01.536765 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:38:01.536718 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pf78n" podStartSLOduration=67.336409667 podStartE2EDuration="1m11.536704471s" totalImagesPullingTime="4.200294804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:38:01.536584898 +0000 UTC m=+71.747430885" watchObservedRunningTime="2026-06-10 12:38:01.536704471 +0000 UTC m=+71.747550462" Jun 10 12:38:01.591259 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:38:01.591209 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fww7p" podStartSLOduration=67.072034462 podStartE2EDuration="1m11.591196061s" totalImagesPullingTime="4.519161599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:38:01.589920646 +0000 UTC m=+71.800766633" watchObservedRunningTime="2026-06-10 12:38:01.591196061 +0000 UTC m=+71.802042039" Jun 10 12:38:28.571305 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:38:28.571274 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:38:28.571794 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:38:28.571390 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:38:28.571794 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:38:28.571448 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls podName:306d7497-fa10-42e3-8c0a-7fb50bf91206 nodeName:}" failed. No retries permitted until 2026-06-10 12:39:32.571433078 +0000 UTC m=+162.782279043 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls") pod "dns-default-vmlx9" (UID: "306d7497-fa10-42e3-8c0a-7fb50bf91206") : secret "dns-default-metrics-tls" not found Jun 10 12:38:32.514365 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:38:32.514333 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pf78n" Jun 10 12:39:00.082408 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:39:00.082351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:39:00.082903 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:39:00.082516 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 12:39:00.082903 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:39:00.082600 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:02.082581288 +0000 UTC m=+252.293427253 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : secret "metrics-daemon-secret" not found Jun 10 12:39:27.702952 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:39:27.702899 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vmlx9" podUID="306d7497-fa10-42e3-8c0a-7fb50bf91206" Jun 10 12:39:27.710785 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:39:27.710757 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmlx9" Jun 10 12:39:29.255118 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:39:29.255041 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:39:32.613040 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:39:32.612996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:39:32.613477 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:39:32.613179 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:39:32.613477 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:39:32.613260 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls podName:306d7497-fa10-42e3-8c0a-7fb50bf91206 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:34.613243103 +0000 UTC m=+284.824089072 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls") pod "dns-default-vmlx9" (UID: "306d7497-fa10-42e3-8c0a-7fb50bf91206") : secret "dns-default-metrics-tls" not found Jun 10 12:39:34.349535 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:39:34.349500 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:39:41.241978 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:39:41.241926 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:40:10.083265 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:10.083227 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:10.146421 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:10.146386 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:45.447091 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.447061 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj"] Jun 10 12:40:45.449892 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.449875 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:45.452549 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.452525 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 10 12:40:45.452970 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.452956 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 10 12:40:45.453919 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.453904 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xt4s4\"" Jun 10 12:40:45.456767 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.456747 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6776685957-2pxv2"] Jun 10 12:40:45.459441 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.459426 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.462581 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.462563 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qfskj\"" Jun 10 12:40:45.463132 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.463115 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 10 12:40:45.463269 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.463255 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 10 12:40:45.463331 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.463297 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 10 12:40:45.463971 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.463953 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 10 12:40:45.464058 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.463953 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 10 12:40:45.466876 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.466861 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 10 12:40:45.481113 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.481067 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6776685957-2pxv2"] Jun 10 12:40:45.481752 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.481728 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj"] Jun 10 12:40:45.509749 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.509726 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:45.509895 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.509753 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.509895 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.509772 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.509895 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.509793 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-default-certificate\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.509895 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.509847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:45.509895 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.509872 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxg6d\" (UniqueName: \"kubernetes.io/projected/5c1b093b-3292-460d-909a-145a7acfc796-kube-api-access-rxg6d\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.510054 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.509977 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-stats-auth\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.547950 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.547916 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw"] Jun 10 12:40:45.550621 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.550605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw" Jun 10 12:40:45.552272 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.552252 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn"] Jun 10 12:40:45.554763 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.554747 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn" Jun 10 12:40:45.557344 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.557326 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mhfrt\"" Jun 10 12:40:45.557621 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.557602 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 10 12:40:45.557700 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.557607 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:40:45.557700 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.557683 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cst84\"" Jun 10 12:40:45.563899 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.563872 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw"] Jun 10 12:40:45.571403 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.571378 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn"] Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.611550 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.611618 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.611653 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p78nd\" (UniqueName: \"kubernetes.io/projected/cdfee373-6546-49da-87ce-2c92603387ca-kube-api-access-p78nd\") pod \"volume-data-source-validator-78dd696465-48nxw\" (UID: \"cdfee373-6546-49da-87ce-2c92603387ca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.611697 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.611742 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-default-certificate\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.611781 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.611812 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxg6d\" (UniqueName: \"kubernetes.io/projected/5c1b093b-3292-460d-909a-145a7acfc796-kube-api-access-rxg6d\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.611877 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-stats-auth\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.611913 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t66gl\" (UniqueName: \"kubernetes.io/projected/8d44bd5a-b1ba-4746-ba80-dbf54523d24c-kube-api-access-t66gl\") pod \"network-check-source-dbd9cf844-lvkbn\" (UID: \"8d44bd5a-b1ba-4746-ba80-dbf54523d24c\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.612719 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:45.613141 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:45.612874 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:46.112856226 +0000 UTC m=+236.323702192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : configmap references non-existent config key: service-ca.crt Jun 10 12:40:45.614358 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:45.613887 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:40:45.614358 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:45.613952 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:46.11393501 +0000 UTC m=+236.324780982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : secret "router-metrics-certs-default" not found Jun 10 12:40:45.614961 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:45.614891 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:40:45.614961 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:45.614973 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert podName:3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:46.114950063 +0000 UTC m=+236.325796043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-mx9nj" (UID: "3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0") : secret "networking-console-plugin-cert" not found Jun 10 12:40:45.617037 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.617016 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-default-certificate\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.617728 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.617709 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-stats-auth\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.628936 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.628908 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxg6d\" (UniqueName: \"kubernetes.io/projected/5c1b093b-3292-460d-909a-145a7acfc796-kube-api-access-rxg6d\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:45.663574 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.663544 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2"] Jun 10 12:40:45.666621 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.666605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:45.669804 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.669786 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-glkqm\"" Jun 10 12:40:45.672189 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.672159 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 10 12:40:45.672390 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.672374 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 10 12:40:45.675743 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.675727 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:40:45.688516 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.688497 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2"] Jun 10 12:40:45.712379 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.712298 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p78nd\" (UniqueName: \"kubernetes.io/projected/cdfee373-6546-49da-87ce-2c92603387ca-kube-api-access-p78nd\") pod \"volume-data-source-validator-78dd696465-48nxw\" (UID: \"cdfee373-6546-49da-87ce-2c92603387ca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw" Jun 10 12:40:45.712379 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.712367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t66gl\" (UniqueName: \"kubernetes.io/projected/8d44bd5a-b1ba-4746-ba80-dbf54523d24c-kube-api-access-t66gl\") pod \"network-check-source-dbd9cf844-lvkbn\" (UID: \"8d44bd5a-b1ba-4746-ba80-dbf54523d24c\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn" Jun 10 12:40:45.723078 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.723054 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t66gl\" (UniqueName: \"kubernetes.io/projected/8d44bd5a-b1ba-4746-ba80-dbf54523d24c-kube-api-access-t66gl\") pod \"network-check-source-dbd9cf844-lvkbn\" (UID: \"8d44bd5a-b1ba-4746-ba80-dbf54523d24c\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn" Jun 10 12:40:45.723258 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.723239 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p78nd\" (UniqueName: \"kubernetes.io/projected/cdfee373-6546-49da-87ce-2c92603387ca-kube-api-access-p78nd\") pod \"volume-data-source-validator-78dd696465-48nxw\" (UID: \"cdfee373-6546-49da-87ce-2c92603387ca\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw" Jun 10 12:40:45.791994 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.791959 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:45.813299 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.813265 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g4db\" (UniqueName: \"kubernetes.io/projected/eaa37133-774c-4327-9470-4200674705df-kube-api-access-7g4db\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:45.813455 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.813378 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:45.854235 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.854201 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:45.859955 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.859933 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw" Jun 10 12:40:45.866707 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.866687 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn" Jun 10 12:40:45.914225 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.914081 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:45.914371 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.914227 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g4db\" (UniqueName: \"kubernetes.io/projected/eaa37133-774c-4327-9470-4200674705df-kube-api-access-7g4db\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:45.914371 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:45.914319 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 10 12:40:45.914493 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:45.914400 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls podName:eaa37133-774c-4327-9470-4200674705df nodeName:}" failed. No retries permitted until 2026-06-10 12:40:46.414375415 +0000 UTC m=+236.625221380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-7vgj2" (UID: "eaa37133-774c-4327-9470-4200674705df") : secret "samples-operator-tls" not found Jun 10 12:40:45.925265 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.924912 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g4db\" (UniqueName: \"kubernetes.io/projected/eaa37133-774c-4327-9470-4200674705df-kube-api-access-7g4db\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:45.993324 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:45.993299 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw"] Jun 10 12:40:45.995623 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:40:45.995589 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdfee373_6546_49da_87ce_2c92603387ca.slice/crio-e7b03f4078f7bee4932bc651596732cfae8921a9e20731791b7d7f0844299e94 WatchSource:0}: Error finding container e7b03f4078f7bee4932bc651596732cfae8921a9e20731791b7d7f0844299e94: Status 404 returned error can't find the container with id e7b03f4078f7bee4932bc651596732cfae8921a9e20731791b7d7f0844299e94 Jun 10 12:40:46.010708 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:46.010684 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn"] Jun 10 12:40:46.013201 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:40:46.013176 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d44bd5a_b1ba_4746_ba80_dbf54523d24c.slice/crio-53fd8314f428a3ce50d7de11a0291e6bb0aa5502dbbf6226d09626e400aedeb0 WatchSource:0}: Error finding container 53fd8314f428a3ce50d7de11a0291e6bb0aa5502dbbf6226d09626e400aedeb0: Status 404 returned error can't find the container with id 53fd8314f428a3ce50d7de11a0291e6bb0aa5502dbbf6226d09626e400aedeb0 Jun 10 12:40:46.115136 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:46.115086 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:46.115288 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:46.115181 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:46.115288 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:46.115201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:46.115288 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:46.115237 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:40:46.115439 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:46.115291 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:40:46.115439 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:46.115304 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert podName:3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:47.115288835 +0000 UTC m=+237.326134801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-mx9nj" (UID: "3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0") : secret "networking-console-plugin-cert" not found Jun 10 12:40:46.115439 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:46.115333 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:47.115322053 +0000 UTC m=+237.326168018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : secret "router-metrics-certs-default" not found Jun 10 12:40:46.115439 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:46.115354 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:47.115337927 +0000 UTC m=+237.326183900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : configmap references non-existent config key: service-ca.crt Jun 10 12:40:46.417641 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:46.417605 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:46.417876 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:46.417722 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 10 12:40:46.417876 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:46.417780 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls podName:eaa37133-774c-4327-9470-4200674705df nodeName:}" failed. No retries permitted until 2026-06-10 12:40:47.417765102 +0000 UTC m=+237.628611067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-7vgj2" (UID: "eaa37133-774c-4327-9470-4200674705df") : secret "samples-operator-tls" not found Jun 10 12:40:46.897173 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:46.897134 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn" event={"ID":"8d44bd5a-b1ba-4746-ba80-dbf54523d24c","Type":"ContainerStarted","Data":"7430fec5256f213ee9d0863998def016e32ed4a94f760d55a92f0e5cbb914a85"} Jun 10 12:40:46.897629 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:46.897196 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn" event={"ID":"8d44bd5a-b1ba-4746-ba80-dbf54523d24c","Type":"ContainerStarted","Data":"53fd8314f428a3ce50d7de11a0291e6bb0aa5502dbbf6226d09626e400aedeb0"} Jun 10 12:40:46.898270 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:46.898240 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw" event={"ID":"cdfee373-6546-49da-87ce-2c92603387ca","Type":"ContainerStarted","Data":"e7b03f4078f7bee4932bc651596732cfae8921a9e20731791b7d7f0844299e94"} Jun 10 12:40:47.123841 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:47.123799 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:47.123841 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:47.123842 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:47.124137 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:47.123874 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:47.124137 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:47.123962 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:40:47.124137 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:47.123971 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:40:47.124137 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:47.123995 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:49.123974648 +0000 UTC m=+239.334820618 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : configmap references non-existent config key: service-ca.crt Jun 10 12:40:47.124137 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:47.124020 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert podName:3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:49.124011145 +0000 UTC m=+239.334857110 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-mx9nj" (UID: "3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0") : secret "networking-console-plugin-cert" not found Jun 10 12:40:47.124137 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:47.124034 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:49.124026572 +0000 UTC m=+239.334872560 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : secret "router-metrics-certs-default" not found Jun 10 12:40:47.427193 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:47.427151 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:47.427390 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:47.427329 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 10 12:40:47.427448 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:47.427416 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls podName:eaa37133-774c-4327-9470-4200674705df nodeName:}" failed. No retries permitted until 2026-06-10 12:40:49.427393477 +0000 UTC m=+239.638239442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-7vgj2" (UID: "eaa37133-774c-4327-9470-4200674705df") : secret "samples-operator-tls" not found Jun 10 12:40:47.902769 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:47.902735 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw" event={"ID":"cdfee373-6546-49da-87ce-2c92603387ca","Type":"ContainerStarted","Data":"4abf310b6b44eb16fa3fa68de077dedf041fc32781774b5cd99367543293094d"} Jun 10 12:40:47.921178 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:47.921127 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-48nxw" podStartSLOduration=1.351814289 podStartE2EDuration="2.921089218s" totalImagesPullingTime="1.569274929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:40:47.919840822 +0000 UTC m=+238.130686808" watchObservedRunningTime="2026-06-10 12:40:47.921089218 +0000 UTC m=+238.131935205" Jun 10 12:40:47.921341 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:47.921201 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-lvkbn" podStartSLOduration=2.921196462 podStartE2EDuration="2.921196462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:40:46.914261067 +0000 UTC m=+237.125107054" watchObservedRunningTime="2026-06-10 12:40:47.921196462 +0000 UTC m=+238.132042449" Jun 10 12:40:47.996242 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:47.996207 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:48.054829 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:48.054795 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:49.141037 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:49.140997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:49.141037 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:49.141042 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:49.141535 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:49.141120 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:49.141535 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:49.141180 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:53.141163454 +0000 UTC m=+243.352009424 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : configmap references non-existent config key: service-ca.crt Jun 10 12:40:49.141535 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:49.141207 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:40:49.141535 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:49.141237 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:40:49.141535 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:49.141250 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:53.141238624 +0000 UTC m=+243.352084589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : secret "router-metrics-certs-default" not found Jun 10 12:40:49.141535 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:49.141279 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert podName:3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0 nodeName:}" failed. No retries permitted until 2026-06-10 12:40:53.141266221 +0000 UTC m=+243.352112187 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-mx9nj" (UID: "3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0") : secret "networking-console-plugin-cert" not found Jun 10 12:40:49.444246 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:49.444138 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:49.444398 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:49.444286 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 10 12:40:49.444398 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:49.444352 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls podName:eaa37133-774c-4327-9470-4200674705df nodeName:}" failed. No retries permitted until 2026-06-10 12:40:53.444337201 +0000 UTC m=+243.655183166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-7vgj2" (UID: "eaa37133-774c-4327-9470-4200674705df") : secret "samples-operator-tls" not found Jun 10 12:40:51.064453 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:51.064424 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:51.257000 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:51.256955 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:51.351248 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:51.351159 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:51.746168 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:51.746052 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:51.858534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:51.858491 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:52.063147 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:52.063092 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:52.159486 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:52.159459 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:52.868963 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:52.868927 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:52.929680 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:52.929642 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:53.170742 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.170652 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:40:53.170742 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.170720 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:53.171170 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.170744 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:40:53.171170 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:53.170819 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:40:53.171170 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:53.170841 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:40:53.171170 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:53.170844 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:01.170822413 +0000 UTC m=+251.381668378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : configmap references non-existent config key: service-ca.crt Jun 10 12:40:53.171170 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:53.170902 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert podName:3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:01.170890357 +0000 UTC m=+251.381736324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-mx9nj" (UID: "3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0") : secret "networking-console-plugin-cert" not found Jun 10 12:40:53.171170 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:53.170915 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:01.170908471 +0000 UTC m=+251.381754436 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : secret "router-metrics-certs-default" not found Jun 10 12:40:53.473215 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.473089 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:40:53.473354 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:53.473252 2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 10 12:40:53.473354 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:40:53.473322 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls podName:eaa37133-774c-4327-9470-4200674705df nodeName:}" failed. No retries permitted until 2026-06-10 12:41:01.473307625 +0000 UTC m=+251.684153590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-7vgj2" (UID: "eaa37133-774c-4327-9470-4200674705df") : secret "samples-operator-tls" not found Jun 10 12:40:53.612894 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.612861 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-6rphg"] Jun 10 12:40:53.615867 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.615852 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.619778 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.619756 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 10 12:40:53.619905 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.619867 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 10 12:40:53.619905 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.619879 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fvvd4\"" Jun 10 12:40:53.621028 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.621011 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 10 12:40:53.623177 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.623151 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 10 12:40:53.631927 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.631906 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-6rphg"] Jun 10 12:40:53.664575 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.664548 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:53.765728 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.765655 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:53.775731 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.775707 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rphms\" (UniqueName: \"kubernetes.io/projected/f139d5d1-7dc6-40f8-9fd0-3454cb73fe42-kube-api-access-rphms\") pod \"service-ca-7d74d578b8-6rphg\" (UID: \"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42\") " pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.775805 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.775773 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f139d5d1-7dc6-40f8-9fd0-3454cb73fe42-signing-key\") pod \"service-ca-7d74d578b8-6rphg\" (UID: \"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42\") " pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.775854 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.775815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f139d5d1-7dc6-40f8-9fd0-3454cb73fe42-signing-cabundle\") pod \"service-ca-7d74d578b8-6rphg\" (UID: \"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42\") " pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.788947 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.788926 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5m4pd_666e8e25-9dde-4db2-a3e2-3e75b5e24059/dns-node-resolver/0.log" Jun 10 12:40:53.876812 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.876780 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rphms\" (UniqueName: \"kubernetes.io/projected/f139d5d1-7dc6-40f8-9fd0-3454cb73fe42-kube-api-access-rphms\") pod \"service-ca-7d74d578b8-6rphg\" (UID: \"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42\") " pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.876928 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.876855 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f139d5d1-7dc6-40f8-9fd0-3454cb73fe42-signing-key\") pod \"service-ca-7d74d578b8-6rphg\" (UID: \"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42\") " pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.876928 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.876887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f139d5d1-7dc6-40f8-9fd0-3454cb73fe42-signing-cabundle\") pod \"service-ca-7d74d578b8-6rphg\" (UID: \"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42\") " pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.877467 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.877449 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f139d5d1-7dc6-40f8-9fd0-3454cb73fe42-signing-cabundle\") pod \"service-ca-7d74d578b8-6rphg\" (UID: \"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42\") " pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.879457 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.879436 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f139d5d1-7dc6-40f8-9fd0-3454cb73fe42-signing-key\") pod \"service-ca-7d74d578b8-6rphg\" (UID: \"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42\") " pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.887514 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.887491 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rphms\" (UniqueName: \"kubernetes.io/projected/f139d5d1-7dc6-40f8-9fd0-3454cb73fe42-kube-api-access-rphms\") pod \"service-ca-7d74d578b8-6rphg\" (UID: \"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42\") " pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:53.923706 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:53.923658 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" Jun 10 12:40:54.054278 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:54.054149 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-6rphg"] Jun 10 12:40:54.057361 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:40:54.057323 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf139d5d1_7dc6_40f8_9fd0_3454cb73fe42.slice/crio-0533a01db205b6d341f4c11e147fff03ba331a1ccddc1e12a8146bec54cf87d3 WatchSource:0}: Error finding container 0533a01db205b6d341f4c11e147fff03ba331a1ccddc1e12a8146bec54cf87d3: Status 404 returned error can't find the container with id 0533a01db205b6d341f4c11e147fff03ba331a1ccddc1e12a8146bec54cf87d3 Jun 10 12:40:54.830928 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:54.830885 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:54.929262 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:54.929223 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" event={"ID":"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42","Type":"ContainerStarted","Data":"0533a01db205b6d341f4c11e147fff03ba331a1ccddc1e12a8146bec54cf87d3"} Jun 10 12:40:54.995518 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:54.995492 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gqpm5_af9f4f53-a664-4c7d-a457-9e0779ad3c16/node-ca/0.log" Jun 10 12:40:55.586829 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:55.586798 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f5d7t_95d21008-75a7-4b76-8a26-ede242d95fd4/serve-healthcheck-canary/0.log" Jun 10 12:40:56.936283 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:56.936240 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" event={"ID":"f139d5d1-7dc6-40f8-9fd0-3454cb73fe42","Type":"ContainerStarted","Data":"c9356aaf31585c7c580ca6f3ee1cac8a71dd32c4d13f92d6a70822973686aafc"} Jun 10 12:40:56.956637 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:56.956583 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-6rphg" podStartSLOduration=2.046351968 podStartE2EDuration="3.956564923s" totalImagesPullingTime="1.910212955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:40:56.954805099 +0000 UTC m=+247.165651086" watchObservedRunningTime="2026-06-10 12:40:56.956564923 +0000 UTC m=+247.167410910" Jun 10 12:40:57.693893 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:57.693848 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:57.819424 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:57.819385 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:58.076714 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:58.076679 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:58.177596 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:58.177565 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:58.979950 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:58.979922 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:40:59.946323 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:40:59.946237 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:00.363001 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:00.362962 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:00.765985 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:00.765900 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:01.156518 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.156460 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:01.237772 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.237728 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:01.237961 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.237782 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:01.237961 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.237856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:41:01.237961 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:01.237902 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:17.237884009 +0000 UTC m=+267.448729978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : configmap references non-existent config key: service-ca.crt Jun 10 12:41:01.238167 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:01.237972 2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:41:01.238167 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:01.237977 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:41:01.238167 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:01.238050 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert podName:3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:17.238028871 +0000 UTC m=+267.448874844 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-mx9nj" (UID: "3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0") : secret "networking-console-plugin-cert" not found Jun 10 12:41:01.238167 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:01.238129 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs podName:5c1b093b-3292-460d-909a-145a7acfc796 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:17.238092218 +0000 UTC m=+267.448938200 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs") pod "router-default-6776685957-2pxv2" (UID: "5c1b093b-3292-460d-909a-145a7acfc796") : secret "router-metrics-certs-default" not found Jun 10 12:41:01.541470 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.541377 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:41:01.544002 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.543965 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eaa37133-774c-4327-9470-4200674705df-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-7vgj2\" (UID: \"eaa37133-774c-4327-9470-4200674705df\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:41:01.559578 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.559546 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:01.577977 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.577952 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-glkqm\"" Jun 10 12:41:01.586521 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.586497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" Jun 10 12:41:01.632028 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.632001 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:01.705737 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.705709 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:01.713753 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.713721 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2"] Jun 10 12:41:01.954951 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.954905 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" event={"ID":"eaa37133-774c-4327-9470-4200674705df","Type":"ContainerStarted","Data":"8a6eee0ca1c9658ce86b1c2d00fe29627c14d335918209d41015e5db02c8a0bd"} Jun 10 12:41:01.976994 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:01.976949 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:02.145758 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:02.145718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:41:02.145954 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:02.145837 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 12:41:02.145954 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:02.145896 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs podName:433d9f02-513b-40d5-b647-9a9a3b4fa4e8 nodeName:}" failed. No retries permitted until 2026-06-10 12:43:04.145878633 +0000 UTC m=+374.356724598 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs") pod "network-metrics-daemon-zlgzd" (UID: "433d9f02-513b-40d5-b647-9a9a3b4fa4e8") : secret "metrics-daemon-secret" not found Jun 10 12:41:02.368245 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:02.368167 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:02.562397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:02.562364 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:02.965755 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:02.965710 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:03.619569 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:03.619492 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:03.749636 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:03.749600 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:03.962069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:03.961977 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" event={"ID":"eaa37133-774c-4327-9470-4200674705df","Type":"ContainerStarted","Data":"ebed7e2a2a8c5046161046e29b7d703fb67f998017d584fcc3bb6c847d08eed0"} Jun 10 12:41:03.962069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:03.962030 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" event={"ID":"eaa37133-774c-4327-9470-4200674705df","Type":"ContainerStarted","Data":"9a72472d086be4e1deeff3a9c858b1a086e0d4e001abb4c0a9ef2a668b38b34d"} Jun 10 12:41:03.981188 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:03.981136 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-7vgj2" podStartSLOduration=17.381668023 podStartE2EDuration="18.981116325s" totalImagesPullingTime="1.599448302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:03.980633394 +0000 UTC m=+254.191479384" watchObservedRunningTime="2026-06-10 12:41:03.981116325 +0000 UTC m=+254.191962303" Jun 10 12:41:04.021552 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:04.021516 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:04.161471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:04.161443 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:04.376345 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:04.376311 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:04.761812 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:04.761732 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:04.963883 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:04.963835 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:05.367596 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:05.367565 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:05.984056 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:05.984019 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:06.169395 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:06.169361 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:06.366660 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:06.366623 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:06.779115 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:06.778996 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:06.963150 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:06.963093 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:07.364948 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:07.364914 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:07.774041 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:07.773965 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:08.176868 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:08.176837 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:08.363643 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:08.363615 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:10.732898 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:10.732859 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:10.804140 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:10.804092 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:16.985616 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:16.985582 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dzbg8"] Jun 10 12:41:16.991961 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:16.991940 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:16.996746 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:16.996699 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 10 12:41:16.996746 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:16.996700 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hlzxg\"" Jun 10 12:41:16.996746 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:16.996740 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 10 12:41:16.996984 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:16.996700 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 10 12:41:16.996984 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:16.996910 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 10 12:41:17.013337 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.013311 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dzbg8"] Jun 10 12:41:17.060383 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.060345 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q9z8\" (UniqueName: \"kubernetes.io/projected/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-kube-api-access-4q9z8\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.060579 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.060409 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-data-volume\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.060579 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.060489 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.060579 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.060530 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-crio-socket\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.060579 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.060564 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.075884 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.075855 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d59c77bf7-bcjl2"] Jun 10 12:41:17.080630 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.080611 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.086168 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.086137 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 10 12:41:17.086329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.086224 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 10 12:41:17.086329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.086299 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gzwg7\"" Jun 10 12:41:17.086834 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.086813 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 10 12:41:17.097112 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.097083 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 10 12:41:17.123976 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.123946 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d59c77bf7-bcjl2"] Jun 10 12:41:17.161409 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-data-volume\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.161409 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161411 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.161661 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161432 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d476b41-3a3c-42e3-aa56-db0425aa0093-registry-certificates\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.161661 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161475 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d476b41-3a3c-42e3-aa56-db0425aa0093-registry-tls\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.161661 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161495 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-crio-socket\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.161661 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161519 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d476b41-3a3c-42e3-aa56-db0425aa0093-ca-trust-extracted\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.161661 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161540 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d476b41-3a3c-42e3-aa56-db0425aa0093-installation-pull-secrets\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.161661 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161587 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d476b41-3a3c-42e3-aa56-db0425aa0093-bound-sa-token\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.161904 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161667 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-crio-socket\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.161904 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161737 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.161904 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161760 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-data-volume\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.161904 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161791 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txnn6\" (UniqueName: \"kubernetes.io/projected/6d476b41-3a3c-42e3-aa56-db0425aa0093-kube-api-access-txnn6\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.161904 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q9z8\" (UniqueName: \"kubernetes.io/projected/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-kube-api-access-4q9z8\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.162243 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161940 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d476b41-3a3c-42e3-aa56-db0425aa0093-image-registry-private-configuration\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.162243 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.161970 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d476b41-3a3c-42e3-aa56-db0425aa0093-trusted-ca\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.162243 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.162041 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.164231 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.164209 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.200286 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.200256 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q9z8\" (UniqueName: \"kubernetes.io/projected/052eabd0-4b09-475f-bdb9-b01d4b8fe0ea-kube-api-access-4q9z8\") pod \"insights-runtime-extractor-dzbg8\" (UID: \"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea\") " pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.249417 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.249324 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:17.262598 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262572 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d476b41-3a3c-42e3-aa56-db0425aa0093-registry-certificates\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.262737 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262605 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:41:17.262737 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d476b41-3a3c-42e3-aa56-db0425aa0093-registry-tls\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.262848 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262737 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d476b41-3a3c-42e3-aa56-db0425aa0093-ca-trust-extracted\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.262848 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262786 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d476b41-3a3c-42e3-aa56-db0425aa0093-installation-pull-secrets\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.262848 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262812 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d476b41-3a3c-42e3-aa56-db0425aa0093-bound-sa-token\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.262848 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txnn6\" (UniqueName: \"kubernetes.io/projected/6d476b41-3a3c-42e3-aa56-db0425aa0093-kube-api-access-txnn6\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.263042 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262875 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:17.263042 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:17.263042 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262941 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d476b41-3a3c-42e3-aa56-db0425aa0093-image-registry-private-configuration\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.263042 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.262984 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d476b41-3a3c-42e3-aa56-db0425aa0093-trusted-ca\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.263461 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.263440 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d476b41-3a3c-42e3-aa56-db0425aa0093-ca-trust-extracted\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.263568 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.263545 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d476b41-3a3c-42e3-aa56-db0425aa0093-registry-certificates\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.263781 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.263743 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c1b093b-3292-460d-909a-145a7acfc796-service-ca-bundle\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:17.263956 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.263942 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d476b41-3a3c-42e3-aa56-db0425aa0093-trusted-ca\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.265788 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.265753 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d476b41-3a3c-42e3-aa56-db0425aa0093-registry-tls\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.265890 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.265844 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-mx9nj\" (UID: \"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:41:17.265963 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.265945 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d476b41-3a3c-42e3-aa56-db0425aa0093-installation-pull-secrets\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.266064 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.266047 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1b093b-3292-460d-909a-145a7acfc796-metrics-certs\") pod \"router-default-6776685957-2pxv2\" (UID: \"5c1b093b-3292-460d-909a-145a7acfc796\") " pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:17.266195 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.266175 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d476b41-3a3c-42e3-aa56-db0425aa0093-image-registry-private-configuration\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.272832 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.272810 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qfskj\"" Jun 10 12:41:17.277359 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.277342 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:17.279850 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.279828 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txnn6\" (UniqueName: \"kubernetes.io/projected/6d476b41-3a3c-42e3-aa56-db0425aa0093-kube-api-access-txnn6\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.280374 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.280358 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d476b41-3a3c-42e3-aa56-db0425aa0093-bound-sa-token\") pod \"image-registry-5d59c77bf7-bcjl2\" (UID: \"6d476b41-3a3c-42e3-aa56-db0425aa0093\") " pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.301395 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.301369 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dzbg8" Jun 10 12:41:17.389807 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.389777 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:17.401561 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.401535 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:17.467043 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.467017 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6776685957-2pxv2"] Jun 10 12:41:17.470595 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:17.470564 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1b093b_3292_460d_909a_145a7acfc796.slice/crio-39b83976c77ec0eeb2ed7b9843f484b99c95897df1bd12d7f871a3eadb713240 WatchSource:0}: Error finding container 39b83976c77ec0eeb2ed7b9843f484b99c95897df1bd12d7f871a3eadb713240: Status 404 returned error can't find the container with id 39b83976c77ec0eeb2ed7b9843f484b99c95897df1bd12d7f871a3eadb713240 Jun 10 12:41:17.482566 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.482539 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dzbg8"] Jun 10 12:41:17.485420 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:17.485395 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052eabd0_4b09_475f_bdb9_b01d4b8fe0ea.slice/crio-1ce4a8b25e22d5c81ee07477524e95b7ebcf3c47d056992c655130d18e04df56 WatchSource:0}: Error finding container 1ce4a8b25e22d5c81ee07477524e95b7ebcf3c47d056992c655130d18e04df56: Status 404 returned error can't find the container with id 1ce4a8b25e22d5c81ee07477524e95b7ebcf3c47d056992c655130d18e04df56 Jun 10 12:41:17.562045 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.562015 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d59c77bf7-bcjl2"] Jun 10 12:41:17.562208 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.562172 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xt4s4\"" Jun 10 12:41:17.567173 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:17.567152 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d476b41_3a3c_42e3_aa56_db0425aa0093.slice/crio-4d6e3eaf9073c09f0b662fd6e64ee09cfb3d5dfdc9f12095f46f427f2764b8b1 WatchSource:0}: Error finding container 4d6e3eaf9073c09f0b662fd6e64ee09cfb3d5dfdc9f12095f46f427f2764b8b1: Status 404 returned error can't find the container with id 4d6e3eaf9073c09f0b662fd6e64ee09cfb3d5dfdc9f12095f46f427f2764b8b1 Jun 10 12:41:17.569471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.569452 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" Jun 10 12:41:17.744463 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:17.744373 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj"] Jun 10 12:41:17.747149 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:17.747119 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8c9935_3c74_4f7f_a0d5_81b1abdfa7c0.slice/crio-5271a84756d6f745d3a714d8293e87a2bb000522273f32066647e1fa5868de31 WatchSource:0}: Error finding container 5271a84756d6f745d3a714d8293e87a2bb000522273f32066647e1fa5868de31: Status 404 returned error can't find the container with id 5271a84756d6f745d3a714d8293e87a2bb000522273f32066647e1fa5868de31 Jun 10 12:41:18.005710 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.005670 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" event={"ID":"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0","Type":"ContainerStarted","Data":"5271a84756d6f745d3a714d8293e87a2bb000522273f32066647e1fa5868de31"} Jun 10 12:41:18.007348 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.007314 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" event={"ID":"6d476b41-3a3c-42e3-aa56-db0425aa0093","Type":"ContainerStarted","Data":"6ede6833951828a671bd7f66a1b94f61ac44053158fe089370550e684ba04a2c"} Jun 10 12:41:18.007486 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.007371 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" event={"ID":"6d476b41-3a3c-42e3-aa56-db0425aa0093","Type":"ContainerStarted","Data":"4d6e3eaf9073c09f0b662fd6e64ee09cfb3d5dfdc9f12095f46f427f2764b8b1"} Jun 10 12:41:18.007486 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.007402 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:18.008957 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.008936 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6776685957-2pxv2" event={"ID":"5c1b093b-3292-460d-909a-145a7acfc796","Type":"ContainerStarted","Data":"1425484b7a1249f5b0c2150d8e054fe0bbaf35759a4911111540518417ddec19"} Jun 10 12:41:18.009073 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.008972 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6776685957-2pxv2" event={"ID":"5c1b093b-3292-460d-909a-145a7acfc796","Type":"ContainerStarted","Data":"39b83976c77ec0eeb2ed7b9843f484b99c95897df1bd12d7f871a3eadb713240"} Jun 10 12:41:18.010415 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.010396 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dzbg8" event={"ID":"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea","Type":"ContainerStarted","Data":"2599ea9497bd6de33cdbfacc68d1d40c4307a0276e6d947ac2f7113e1a4c895f"} Jun 10 12:41:18.010493 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.010435 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dzbg8" event={"ID":"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea","Type":"ContainerStarted","Data":"1ce4a8b25e22d5c81ee07477524e95b7ebcf3c47d056992c655130d18e04df56"} Jun 10 12:41:18.037210 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.037149 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" podStartSLOduration=1.03712647 podStartE2EDuration="1.03712647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:41:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:18.035381982 +0000 UTC m=+268.246227970" watchObservedRunningTime="2026-06-10 12:41:18.03712647 +0000 UTC m=+268.247972461" Jun 10 12:41:18.070830 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.070760 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6776685957-2pxv2" podStartSLOduration=33.070739531 podStartE2EDuration="33.070739531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:18.069720534 +0000 UTC m=+268.280566523" watchObservedRunningTime="2026-06-10 12:41:18.070739531 +0000 UTC m=+268.281585520" Jun 10 12:41:18.278269 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.278243 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:18.280565 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:18.280543 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:19.017344 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:19.017297 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dzbg8" event={"ID":"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea","Type":"ContainerStarted","Data":"3b628abca36be541f823a9ab39eb4b80d8d1f1adcf6505f8838860f1aebe96b8"} Jun 10 12:41:19.017760 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:19.017727 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:19.019059 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:19.018996 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6776685957-2pxv2" Jun 10 12:41:20.022591 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:20.022563 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" event={"ID":"3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0","Type":"ContainerStarted","Data":"5fe99ccaf8c5e888d5e30ffc99ccc947f4ab707a6d554f8f3bf395e589ff2144"} Jun 10 12:41:20.039831 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:20.039786 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-mx9nj" podStartSLOduration=33.827762778 podStartE2EDuration="35.039769186s" totalImagesPullingTime="1.212006408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:20.039317936 +0000 UTC m=+270.250163922" watchObservedRunningTime="2026-06-10 12:41:20.039769186 +0000 UTC m=+270.250615172" Jun 10 12:41:20.145701 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:20.145669 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:20.246470 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:20.246381 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:20.369420 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:20.369387 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:20.458581 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:20.458540 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:20.544037 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:20.543946 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:21.027256 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:21.027216 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dzbg8" event={"ID":"052eabd0-4b09-475f-bdb9-b01d4b8fe0ea","Type":"ContainerStarted","Data":"96ae0846198ea3106bd17e8bc1ab3a2e6e3f938ed1267675cf7f45332b9024b5"} Jun 10 12:41:21.053011 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:21.050154 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dzbg8" podStartSLOduration=2.812902225 podStartE2EDuration="5.050126549s" totalImagesPullingTime="2.237224324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:41:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:21.049455623 +0000 UTC m=+271.260301611" watchObservedRunningTime="2026-06-10 12:41:21.050126549 +0000 UTC m=+271.260972533" Jun 10 12:41:21.273255 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:21.273223 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:23.573739 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:23.573705 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:23.650398 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:23.650367 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:23.758952 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:23.758919 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:25.512267 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:25.512236 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:27.791931 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:27.791890 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:27.998500 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:27.998466 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:28.056507 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.056472 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx"] Jun 10 12:41:28.061641 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.061621 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.067066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.066499 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 10 12:41:28.067066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.066544 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 10 12:41:28.067066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.066649 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 10 12:41:28.067066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.066733 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 10 12:41:28.067066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.066863 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-sxdgj\"" Jun 10 12:41:28.067066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.066924 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 10 12:41:28.067066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.067043 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 10 12:41:28.073571 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.073551 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx"] Jun 10 12:41:28.113204 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.113171 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:28.144437 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.144387 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2d630daa-7080-4ae1-b427-0a901405386a-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.144629 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.144469 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfffx\" (UniqueName: \"kubernetes.io/projected/2d630daa-7080-4ae1-b427-0a901405386a-kube-api-access-lfffx\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.144629 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.144508 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.144629 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.144554 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d630daa-7080-4ae1-b427-0a901405386a-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.144629 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.144595 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.144629 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.144621 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.168133 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.166728 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cjz24"] Jun 10 12:41:28.170652 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.170625 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.174283 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.174263 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 10 12:41:28.174452 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.174434 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 10 12:41:28.174542 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.174261 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8wwkk\"" Jun 10 12:41:28.174542 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.174294 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 10 12:41:28.219354 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.219305 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:28.245416 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245384 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2d630daa-7080-4ae1-b427-0a901405386a-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.245569 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245434 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-tls\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.245569 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245460 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-wtmp\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.245569 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245557 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.245746 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245596 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfffx\" (UniqueName: \"kubernetes.io/projected/2d630daa-7080-4ae1-b427-0a901405386a-kube-api-access-lfffx\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.245746 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245672 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-accelerators-collector-config\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.245746 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245714 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.245893 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245744 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccb06a3a-3837-4662-bbfc-a2b80b648493-metrics-client-ca\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.245893 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-textfile\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.245893 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245814 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2d630daa-7080-4ae1-b427-0a901405386a-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.245893 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245816 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccb06a3a-3837-4662-bbfc-a2b80b648493-sys\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.245893 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245871 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ccb06a3a-3837-4662-bbfc-a2b80b648493-root\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.246181 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245917 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d630daa-7080-4ae1-b427-0a901405386a-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.246181 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn7wf\" (UniqueName: \"kubernetes.io/projected/ccb06a3a-3837-4662-bbfc-a2b80b648493-kube-api-access-wn7wf\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.246181 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.245990 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.246181 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.246017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.246558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.246536 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.246680 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:28.246540 2582 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 10 12:41:28.246770 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.246747 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d630daa-7080-4ae1-b427-0a901405386a-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.246872 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:28.246810 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-tls podName:2d630daa-7080-4ae1-b427-0a901405386a nodeName:}" failed. No retries permitted until 2026-06-10 12:41:28.746790158 +0000 UTC m=+278.957636126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-tls") pod "kube-state-metrics-57bbf8bfb5-pd7jx" (UID: "2d630daa-7080-4ae1-b427-0a901405386a") : secret "kube-state-metrics-tls" not found Jun 10 12:41:28.249039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.249015 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.255875 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.255843 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfffx\" (UniqueName: \"kubernetes.io/projected/2d630daa-7080-4ae1-b427-0a901405386a-kube-api-access-lfffx\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.312651 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.312573 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:28.346634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.346595 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccb06a3a-3837-4662-bbfc-a2b80b648493-sys\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.346634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.346645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ccb06a3a-3837-4662-bbfc-a2b80b648493-root\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.346836 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.346679 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn7wf\" (UniqueName: \"kubernetes.io/projected/ccb06a3a-3837-4662-bbfc-a2b80b648493-kube-api-access-wn7wf\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.346836 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.346720 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccb06a3a-3837-4662-bbfc-a2b80b648493-sys\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.346836 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.346823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-tls\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.346934 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.346852 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-wtmp\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.346934 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.346916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.346993 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.346970 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-accelerators-collector-config\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.347025 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.346998 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccb06a3a-3837-4662-bbfc-a2b80b648493-metrics-client-ca\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.347057 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.347028 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-textfile\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.347400 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.347383 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-textfile\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.347486 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.347468 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ccb06a3a-3837-4662-bbfc-a2b80b648493-root\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.347865 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:28.347850 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 10 12:41:28.347926 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:28.347916 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-tls podName:ccb06a3a-3837-4662-bbfc-a2b80b648493 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:28.847899323 +0000 UTC m=+279.058745295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-tls") pod "node-exporter-cjz24" (UID: "ccb06a3a-3837-4662-bbfc-a2b80b648493") : secret "node-exporter-tls" not found Jun 10 12:41:28.348207 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.348189 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-wtmp\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.349022 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.348956 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-accelerators-collector-config\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.349022 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.348965 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccb06a3a-3837-4662-bbfc-a2b80b648493-metrics-client-ca\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.350407 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.350378 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.357057 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.357033 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn7wf\" (UniqueName: \"kubernetes.io/projected/ccb06a3a-3837-4662-bbfc-a2b80b648493-kube-api-access-wn7wf\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.414659 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.414617 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:28.543714 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.543661 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:28.608217 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.608114 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:28.749379 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.749339 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:28.749583 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:28.749509 2582 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 10 12:41:28.749644 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:28.749586 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-tls podName:2d630daa-7080-4ae1-b427-0a901405386a nodeName:}" failed. No retries permitted until 2026-06-10 12:41:29.749566743 +0000 UTC m=+279.960412708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-tls") pod "kube-state-metrics-57bbf8bfb5-pd7jx" (UID: "2d630daa-7080-4ae1-b427-0a901405386a") : secret "kube-state-metrics-tls" not found Jun 10 12:41:28.850495 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.850457 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-tls\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:28.852928 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:28.852899 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ccb06a3a-3837-4662-bbfc-a2b80b648493-node-exporter-tls\") pod \"node-exporter-cjz24\" (UID: \"ccb06a3a-3837-4662-bbfc-a2b80b648493\") " pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:29.062757 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.062726 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:29.080730 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.080710 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cjz24" Jun 10 12:41:29.092968 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:29.092937 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb06a3a_3837_4662_bbfc_a2b80b648493.slice/crio-035fb7de68659b9bd8066e4654cb9e3defa643100969e1dc3ef556b9bdf5b2c4 WatchSource:0}: Error finding container 035fb7de68659b9bd8066e4654cb9e3defa643100969e1dc3ef556b9bdf5b2c4: Status 404 returned error can't find the container with id 035fb7de68659b9bd8066e4654cb9e3defa643100969e1dc3ef556b9bdf5b2c4 Jun 10 12:41:29.159115 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.159070 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:29.181445 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.181417 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:41:29.184895 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.184880 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.187875 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.187841 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 10 12:41:29.187875 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.187858 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 10 12:41:29.188034 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.188005 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-shmlb\"" Jun 10 12:41:29.188034 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.188021 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 10 12:41:29.188163 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.188075 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 10 12:41:29.188220 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.188152 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 10 12:41:29.188277 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.188236 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 10 12:41:29.188484 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.188469 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 10 12:41:29.188536 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.188510 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 10 12:41:29.188669 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.188656 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 10 12:41:29.205271 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.205248 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:41:29.242043 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.242015 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:29.253773 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.253751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zb5p\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-kube-api-access-9zb5p\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.253909 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.253782 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.253909 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.253825 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.253909 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.253847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.253909 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.253889 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-config-volume\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.254064 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.253919 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.254064 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.253936 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-config-out\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.254064 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.253953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.254064 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.253979 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.254064 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.254019 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.254064 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.254043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.254301 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.254088 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-web-config\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.254301 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.254143 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355352 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355270 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355352 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355299 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-config-out\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355352 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355352 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355348 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355691 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355409 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355691 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355449 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355691 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355473 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-web-config\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355691 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355691 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zb5p\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-kube-api-access-9zb5p\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.355691 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:29.355567 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-alertmanager-trusted-ca-bundle podName:1da94685-3c98-4177-8917-027f02be0785 nodeName:}" failed. No retries permitted until 2026-06-10 12:41:29.855544385 +0000 UTC m=+280.066390368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "1da94685-3c98-4177-8917-027f02be0785") : configmap references non-existent config key: ca-bundle.crt Jun 10 12:41:29.355691 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355597 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.356038 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355727 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.356038 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355755 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.356038 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355778 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.356038 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.355786 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-config-volume\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.359003 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.358470 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-config-out\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.359003 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.358685 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.359003 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.358912 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.359003 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.358970 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.359003 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.358970 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.359533 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.359017 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.359533 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.359275 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.359533 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.359359 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-web-config\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.359533 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.359381 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.360874 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.360852 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-config-volume\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.370087 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.370063 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zb5p\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-kube-api-access-9zb5p\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.760087 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.759997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:29.762897 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.762865 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d630daa-7080-4ae1-b427-0a901405386a-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-pd7jx\" (UID: \"2d630daa-7080-4ae1-b427-0a901405386a\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:29.861252 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.861217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.862246 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.862218 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:29.872409 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:29.872200 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" Jun 10 12:41:30.024567 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:30.024514 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx"] Jun 10 12:41:30.026988 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:30.026955 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d630daa_7080_4ae1_b427_0a901405386a.slice/crio-7cbf698dad9b84fb18fe8cceb547d0679d8951d3dda03718c54d025c2c6e25c1 WatchSource:0}: Error finding container 7cbf698dad9b84fb18fe8cceb547d0679d8951d3dda03718c54d025c2c6e25c1: Status 404 returned error can't find the container with id 7cbf698dad9b84fb18fe8cceb547d0679d8951d3dda03718c54d025c2c6e25c1 Jun 10 12:41:30.059135 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:30.059083 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjz24" event={"ID":"ccb06a3a-3837-4662-bbfc-a2b80b648493","Type":"ContainerStarted","Data":"da8e145205aed548bf5964119c6a6cd438ac796d060644cfbeba666fdb8bc8d7"} Jun 10 12:41:30.059326 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:30.059172 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjz24" event={"ID":"ccb06a3a-3837-4662-bbfc-a2b80b648493","Type":"ContainerStarted","Data":"035fb7de68659b9bd8066e4654cb9e3defa643100969e1dc3ef556b9bdf5b2c4"} Jun 10 12:41:30.060423 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:30.060345 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" event={"ID":"2d630daa-7080-4ae1-b427-0a901405386a","Type":"ContainerStarted","Data":"7cbf698dad9b84fb18fe8cceb547d0679d8951d3dda03718c54d025c2c6e25c1"} Jun 10 12:41:30.094419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:30.094384 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:41:30.251084 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:30.248256 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:41:30.711980 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:30.711936 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vmlx9" podUID="306d7497-fa10-42e3-8c0a-7fb50bf91206" Jun 10 12:41:31.054603 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:31.054527 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:31.067239 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:31.067199 2582 generic.go:356] "Generic (PLEG): container finished" podID="ccb06a3a-3837-4662-bbfc-a2b80b648493" containerID="da8e145205aed548bf5964119c6a6cd438ac796d060644cfbeba666fdb8bc8d7" exitCode=0 Jun 10 12:41:31.067402 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:31.067281 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjz24" event={"ID":"ccb06a3a-3837-4662-bbfc-a2b80b648493","Type":"ContainerDied","Data":"da8e145205aed548bf5964119c6a6cd438ac796d060644cfbeba666fdb8bc8d7"} Jun 10 12:41:31.069322 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:31.069253 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmlx9" Jun 10 12:41:31.069723 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:31.069250 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerStarted","Data":"2d87f748ac08b4daa7a20ac31be44c9fe4a00ff00e399f177d91909b7ac06024"} Jun 10 12:41:31.128923 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:31.128886 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:31.499436 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:31.499373 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:31.607950 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:31.607914 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:32.076224 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.076193 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjz24" event={"ID":"ccb06a3a-3837-4662-bbfc-a2b80b648493","Type":"ContainerStarted","Data":"7759af9b2ae0a5472ffb27569d3c66e2d6e214c5cb88116c5208a4869bf66b6c"} Jun 10 12:41:32.076772 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.076262 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjz24" event={"ID":"ccb06a3a-3837-4662-bbfc-a2b80b648493","Type":"ContainerStarted","Data":"e322010af24e4d706398b7726b7101d73d130f2c05e7464a56512e0dd1da115e"} Jun 10 12:41:32.077829 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.077805 2582 generic.go:356] "Generic (PLEG): container finished" podID="1da94685-3c98-4177-8917-027f02be0785" containerID="d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b" exitCode=0 Jun 10 12:41:32.077967 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.077943 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerDied","Data":"d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b"} Jun 10 12:41:32.080386 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.080348 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" event={"ID":"2d630daa-7080-4ae1-b427-0a901405386a","Type":"ContainerStarted","Data":"edf5052a51a22b7066da69049041c804519190fc90abb0725d160d6625ada094"} Jun 10 12:41:32.080506 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.080396 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" event={"ID":"2d630daa-7080-4ae1-b427-0a901405386a","Type":"ContainerStarted","Data":"d77d1042a9903f2b2b56b58c415b00b72b86a9842e75f3538fa220674a4ed0e0"} Jun 10 12:41:32.080506 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.080435 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" event={"ID":"2d630daa-7080-4ae1-b427-0a901405386a","Type":"ContainerStarted","Data":"96143f96de297d3e54efea6d9fcfd720ab4f1059f86ac11fb1aa6f3929fe6fb5"} Jun 10 12:41:32.137429 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.137353 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cjz24" podStartSLOduration=3.221806688 podStartE2EDuration="4.137326456s" totalImagesPullingTime="840.420589ms" totalInitContainerRuntime="75.099179ms" isStatefulPod=true podCreationTimestamp="2026-06-10 12:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:32.103852054 +0000 UTC m=+282.314698040" watchObservedRunningTime="2026-06-10 12:41:32.137326456 +0000 UTC m=+282.348172443" Jun 10 12:41:32.137852 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.137815 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-pd7jx" podStartSLOduration=2.383171228 podStartE2EDuration="4.137799572s" totalImagesPullingTime="1.754628344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:32.134865509 +0000 UTC m=+282.345711529" watchObservedRunningTime="2026-06-10 12:41:32.137799572 +0000 UTC m=+282.348645559" Jun 10 12:41:32.420010 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.419963 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-867d7ffb85-s2wvx"] Jun 10 12:41:32.424399 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.424374 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.428479 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.428449 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 10 12:41:32.428627 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.428480 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-32mrij2g5cc7e\"" Jun 10 12:41:32.428627 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.428543 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9ndqv\"" Jun 10 12:41:32.429270 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.429250 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 10 12:41:32.429911 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.429884 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 10 12:41:32.430015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.429996 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 10 12:41:32.441985 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.441958 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-867d7ffb85-s2wvx"] Jun 10 12:41:32.486267 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.486226 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc283677-d4ee-4fab-a89f-e09a25a8cef6-client-ca-bundle\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.486421 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.486272 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dc283677-d4ee-4fab-a89f-e09a25a8cef6-secret-metrics-server-client-certs\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.486421 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.486355 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dc283677-d4ee-4fab-a89f-e09a25a8cef6-secret-metrics-server-tls\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.486421 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.486411 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc283677-d4ee-4fab-a89f-e09a25a8cef6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.486558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.486443 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dc283677-d4ee-4fab-a89f-e09a25a8cef6-audit-log\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.486558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.486478 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvck6\" (UniqueName: \"kubernetes.io/projected/dc283677-d4ee-4fab-a89f-e09a25a8cef6-kube-api-access-jvck6\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.486558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.486541 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dc283677-d4ee-4fab-a89f-e09a25a8cef6-metrics-server-audit-profiles\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.587022 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.586971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dc283677-d4ee-4fab-a89f-e09a25a8cef6-metrics-server-audit-profiles\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.587203 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.587077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc283677-d4ee-4fab-a89f-e09a25a8cef6-client-ca-bundle\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.587203 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.587132 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dc283677-d4ee-4fab-a89f-e09a25a8cef6-secret-metrics-server-client-certs\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.587203 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.587192 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dc283677-d4ee-4fab-a89f-e09a25a8cef6-secret-metrics-server-tls\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.587396 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.587221 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc283677-d4ee-4fab-a89f-e09a25a8cef6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.587462 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.587436 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dc283677-d4ee-4fab-a89f-e09a25a8cef6-audit-log\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.587514 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.587490 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvck6\" (UniqueName: \"kubernetes.io/projected/dc283677-d4ee-4fab-a89f-e09a25a8cef6-kube-api-access-jvck6\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.587905 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.587885 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc283677-d4ee-4fab-a89f-e09a25a8cef6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.587972 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.587881 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dc283677-d4ee-4fab-a89f-e09a25a8cef6-audit-log\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.588283 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.588253 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dc283677-d4ee-4fab-a89f-e09a25a8cef6-metrics-server-audit-profiles\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.590447 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.590425 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc283677-d4ee-4fab-a89f-e09a25a8cef6-client-ca-bundle\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.590559 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.590507 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dc283677-d4ee-4fab-a89f-e09a25a8cef6-secret-metrics-server-client-certs\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.590780 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.590760 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dc283677-d4ee-4fab-a89f-e09a25a8cef6-secret-metrics-server-tls\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.602265 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.602239 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvck6\" (UniqueName: \"kubernetes.io/projected/dc283677-d4ee-4fab-a89f-e09a25a8cef6-kube-api-access-jvck6\") pod \"metrics-server-867d7ffb85-s2wvx\" (UID: \"dc283677-d4ee-4fab-a89f-e09a25a8cef6\") " pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.734576 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.734062 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:32.776147 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.776081 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:32.792987 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.791614 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp"] Jun 10 12:41:32.798238 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.797165 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:32.802510 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.802276 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gkvrr\"" Jun 10 12:41:32.802632 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.802549 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 10 12:41:32.809309 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.809088 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp"] Jun 10 12:41:32.891892 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.891846 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsrxj\" (UniqueName: \"kubernetes.io/projected/9f6621df-fea0-4d8b-95a9-b48be65a38cd-kube-api-access-qsrxj\") pod \"monitoring-plugin-76b68cc874-p6mrp\" (UID: \"9f6621df-fea0-4d8b-95a9-b48be65a38cd\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:32.892074 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.891897 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9f6621df-fea0-4d8b-95a9-b48be65a38cd-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-p6mrp\" (UID: \"9f6621df-fea0-4d8b-95a9-b48be65a38cd\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:32.922206 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.922170 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-867d7ffb85-s2wvx"] Jun 10 12:41:32.925831 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:32.925756 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc283677_d4ee_4fab_a89f_e09a25a8cef6.slice/crio-06b2e9dcc25d9d42ad9d92cbb664fc31d800e732f501f883f3dae7850a2e348a WatchSource:0}: Error finding container 06b2e9dcc25d9d42ad9d92cbb664fc31d800e732f501f883f3dae7850a2e348a: Status 404 returned error can't find the container with id 06b2e9dcc25d9d42ad9d92cbb664fc31d800e732f501f883f3dae7850a2e348a Jun 10 12:41:32.992518 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.992424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsrxj\" (UniqueName: \"kubernetes.io/projected/9f6621df-fea0-4d8b-95a9-b48be65a38cd-kube-api-access-qsrxj\") pod \"monitoring-plugin-76b68cc874-p6mrp\" (UID: \"9f6621df-fea0-4d8b-95a9-b48be65a38cd\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:32.992518 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:32.992477 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9f6621df-fea0-4d8b-95a9-b48be65a38cd-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-p6mrp\" (UID: \"9f6621df-fea0-4d8b-95a9-b48be65a38cd\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:32.992747 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:32.992642 2582 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 10 12:41:32.992747 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:32.992703 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f6621df-fea0-4d8b-95a9-b48be65a38cd-monitoring-plugin-cert podName:9f6621df-fea0-4d8b-95a9-b48be65a38cd nodeName:}" failed. No retries permitted until 2026-06-10 12:41:33.492681659 +0000 UTC m=+283.703527626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/9f6621df-fea0-4d8b-95a9-b48be65a38cd-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-p6mrp" (UID: "9f6621df-fea0-4d8b-95a9-b48be65a38cd") : secret "monitoring-plugin-cert" not found Jun 10 12:41:33.003965 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:33.003926 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsrxj\" (UniqueName: \"kubernetes.io/projected/9f6621df-fea0-4d8b-95a9-b48be65a38cd-kube-api-access-qsrxj\") pod \"monitoring-plugin-76b68cc874-p6mrp\" (UID: \"9f6621df-fea0-4d8b-95a9-b48be65a38cd\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:33.086217 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:33.086172 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" event={"ID":"dc283677-d4ee-4fab-a89f-e09a25a8cef6","Type":"ContainerStarted","Data":"06b2e9dcc25d9d42ad9d92cbb664fc31d800e732f501f883f3dae7850a2e348a"} Jun 10 12:41:33.140164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:33.140120 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:33.496564 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:33.496531 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9f6621df-fea0-4d8b-95a9-b48be65a38cd-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-p6mrp\" (UID: \"9f6621df-fea0-4d8b-95a9-b48be65a38cd\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:33.499772 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:33.499743 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9f6621df-fea0-4d8b-95a9-b48be65a38cd-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-p6mrp\" (UID: \"9f6621df-fea0-4d8b-95a9-b48be65a38cd\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:33.716913 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:33.716886 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:33.872537 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:33.872442 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp"] Jun 10 12:41:33.880850 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:33.880821 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6621df_fea0_4d8b_95a9_b48be65a38cd.slice/crio-6cf9d7ae82639552e5a9943107567ddf5104f448cfc212c6ebef84966e7b8c61 WatchSource:0}: Error finding container 6cf9d7ae82639552e5a9943107567ddf5104f448cfc212c6ebef84966e7b8c61: Status 404 returned error can't find the container with id 6cf9d7ae82639552e5a9943107567ddf5104f448cfc212c6ebef84966e7b8c61 Jun 10 12:41:34.094397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.094355 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerStarted","Data":"30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b"} Jun 10 12:41:34.094809 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.094442 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerStarted","Data":"1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743"} Jun 10 12:41:34.094809 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.094486 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerStarted","Data":"56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1"} Jun 10 12:41:34.094809 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.094524 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerStarted","Data":"9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3"} Jun 10 12:41:34.094809 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.094563 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerStarted","Data":"caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891"} Jun 10 12:41:34.095618 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.095595 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" event={"ID":"9f6621df-fea0-4d8b-95a9-b48be65a38cd","Type":"ContainerStarted","Data":"6cf9d7ae82639552e5a9943107567ddf5104f448cfc212c6ebef84966e7b8c61"} Jun 10 12:41:34.345294 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.345209 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:34.704983 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.704909 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:41:34.707430 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.707402 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/306d7497-fa10-42e3-8c0a-7fb50bf91206-metrics-tls\") pod \"dns-default-vmlx9\" (UID: \"306d7497-fa10-42e3-8c0a-7fb50bf91206\") " pod="openshift-dns/dns-default-vmlx9" Jun 10 12:41:34.973343 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.973219 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hpqfd\"" Jun 10 12:41:34.980592 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:34.980510 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmlx9" Jun 10 12:41:35.106216 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:35.106129 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerStarted","Data":"38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79"} Jun 10 12:41:35.108334 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:35.108293 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" event={"ID":"dc283677-d4ee-4fab-a89f-e09a25a8cef6","Type":"ContainerStarted","Data":"f145d3ebe280219e547fbb7bc3047dde56229043278efc2a4856cbbd9a240b6e"} Jun 10 12:41:35.148510 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:35.148485 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vmlx9"] Jun 10 12:41:35.150887 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:35.150857 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306d7497_fa10_42e3_8c0a_7fb50bf91206.slice/crio-d5b19e3a330f0c0c776ab10325b536038d4b35a8119ce2ef4a79cc359e252c4d WatchSource:0}: Error finding container d5b19e3a330f0c0c776ab10325b536038d4b35a8119ce2ef4a79cc359e252c4d: Status 404 returned error can't find the container with id d5b19e3a330f0c0c776ab10325b536038d4b35a8119ce2ef4a79cc359e252c4d Jun 10 12:41:35.174188 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:35.174130 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.205960141 podStartE2EDuration="6.174090874s" totalImagesPullingTime="3.947289493s" totalInitContainerRuntime="20.84124ms" isStatefulPod=false podCreationTimestamp="2026-06-10 12:41:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:35.17066996 +0000 UTC m=+285.381515949" watchObservedRunningTime="2026-06-10 12:41:35.174090874 +0000 UTC m=+285.384936839" Jun 10 12:41:35.219975 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:35.219909 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" podStartSLOduration=1.299547823 podStartE2EDuration="3.219889799s" totalImagesPullingTime="1.920341976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:35.21830259 +0000 UTC m=+285.429148578" watchObservedRunningTime="2026-06-10 12:41:35.219889799 +0000 UTC m=+285.430735784" Jun 10 12:41:36.112442 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:36.112405 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmlx9" event={"ID":"306d7497-fa10-42e3-8c0a-7fb50bf91206","Type":"ContainerStarted","Data":"d5b19e3a330f0c0c776ab10325b536038d4b35a8119ce2ef4a79cc359e252c4d"} Jun 10 12:41:36.113716 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:36.113688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" event={"ID":"9f6621df-fea0-4d8b-95a9-b48be65a38cd","Type":"ContainerStarted","Data":"4500631954ca73ca776e2befa3534ac9edc7a318d2e2aa0682eb9bac25327812"} Jun 10 12:41:36.135480 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:36.135427 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" podStartSLOduration=2.299869039 podStartE2EDuration="4.135411395s" totalImagesPullingTime="1.835542356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:36.13335168 +0000 UTC m=+286.344197667" watchObservedRunningTime="2026-06-10 12:41:36.135411395 +0000 UTC m=+286.346257382" Jun 10 12:41:37.117371 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:37.117343 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:37.123712 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:37.123603 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-p6mrp" Jun 10 12:41:37.394236 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:37.394148 2582 patch_prober.go:28] interesting pod/image-registry-5d59c77bf7-bcjl2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 10 12:41:37.394236 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:37.394208 2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" podUID="6d476b41-3a3c-42e3-aa56-db0425aa0093" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 10 12:41:38.122741 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:38.122704 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmlx9" event={"ID":"306d7497-fa10-42e3-8c0a-7fb50bf91206","Type":"ContainerStarted","Data":"79fd971371518dd470b2ed96dd607c7475d6b9a41b750e85e379ca184c430e8d"} Jun 10 12:41:38.123179 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:38.122768 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmlx9" event={"ID":"306d7497-fa10-42e3-8c0a-7fb50bf91206","Type":"ContainerStarted","Data":"458554d4a066183a9c3eb7519bd4d418cdfcff135c7024129924976eede09ba2"} Jun 10 12:41:38.123179 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:38.122852 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vmlx9" Jun 10 12:41:38.145897 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:38.145848 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vmlx9" podStartSLOduration=252.196360714 podStartE2EDuration="4m14.145831148s" totalImagesPullingTime="1.949470434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:38.144692155 +0000 UTC m=+288.355538142" watchObservedRunningTime="2026-06-10 12:41:38.145831148 +0000 UTC m=+288.356677161" Jun 10 12:41:39.023192 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:39.023155 2582 patch_prober.go:28] interesting pod/image-registry-5d59c77bf7-bcjl2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 10 12:41:39.023363 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:39.023199 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" podUID="6d476b41-3a3c-42e3-aa56-db0425aa0093" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 10 12:41:41.760726 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.760679 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-f8995"] Jun 10 12:41:41.765017 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.764998 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-f8995" Jun 10 12:41:41.769231 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.769206 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 10 12:41:41.769336 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.769267 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 10 12:41:41.770766 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.770751 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-c66fk\"" Jun 10 12:41:41.780730 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.780702 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:41.781532 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.781508 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-f8995"] Jun 10 12:41:41.863448 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.863402 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wl66\" (UniqueName: \"kubernetes.io/projected/066f8f06-a2e4-4703-9e42-ce8067800a2c-kube-api-access-6wl66\") pod \"downloads-6657875f97-f8995\" (UID: \"066f8f06-a2e4-4703-9e42-ce8067800a2c\") " pod="openshift-console/downloads-6657875f97-f8995" Jun 10 12:41:41.870524 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.870488 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:41.964837 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.964805 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wl66\" (UniqueName: \"kubernetes.io/projected/066f8f06-a2e4-4703-9e42-ce8067800a2c-kube-api-access-6wl66\") pod \"downloads-6657875f97-f8995\" (UID: \"066f8f06-a2e4-4703-9e42-ce8067800a2c\") " pod="openshift-console/downloads-6657875f97-f8995" Jun 10 12:41:41.972248 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.972219 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:41.975020 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:41.974998 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wl66\" (UniqueName: \"kubernetes.io/projected/066f8f06-a2e4-4703-9e42-ce8067800a2c-kube-api-access-6wl66\") pod \"downloads-6657875f97-f8995\" (UID: \"066f8f06-a2e4-4703-9e42-ce8067800a2c\") " pod="openshift-console/downloads-6657875f97-f8995" Jun 10 12:41:42.074130 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:42.074082 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-f8995" Jun 10 12:41:42.206029 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:42.205999 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-f8995"] Jun 10 12:41:42.208932 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:42.208906 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066f8f06_a2e4_4703_9e42_ce8067800a2c.slice/crio-7e89a24bb7f83df69b2c7d2f495e55e144c7380fbeb0e10754403af70f5ab2aa WatchSource:0}: Error finding container 7e89a24bb7f83df69b2c7d2f495e55e144c7380fbeb0e10754403af70f5ab2aa: Status 404 returned error can't find the container with id 7e89a24bb7f83df69b2c7d2f495e55e144c7380fbeb0e10754403af70f5ab2aa Jun 10 12:41:42.249042 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:42.249013 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:42.428467 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:42.428384 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:42.530198 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:42.530165 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:42.641847 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:42.641819 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:42.868754 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:42.868713 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:42.927337 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:42.927311 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:43.144435 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:43.144359 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-f8995" event={"ID":"066f8f06-a2e4-4703-9e42-ce8067800a2c","Type":"ContainerStarted","Data":"7e89a24bb7f83df69b2c7d2f495e55e144c7380fbeb0e10754403af70f5ab2aa"} Jun 10 12:41:43.241737 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:43.241703 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:43.540988 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:43.540911 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:43.840444 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:43.840414 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:44.242752 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:41:44.242659 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-zlgzd" podUID="433d9f02-513b-40d5-b647-9a9a3b4fa4e8" Jun 10 12:41:45.060647 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:45.060590 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:45.889119 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:45.889071 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:45.991969 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:45.991912 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:46.090026 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:46.089991 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:46.772991 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:46.772955 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:46.825408 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:46.825376 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:47.395338 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:47.395299 2582 patch_prober.go:28] interesting pod/image-registry-5d59c77bf7-bcjl2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 10 12:41:47.395826 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:47.395364 2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" podUID="6d476b41-3a3c-42e3-aa56-db0425aa0093" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 10 12:41:48.129608 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:48.129569 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vmlx9" Jun 10 12:41:49.023142 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:49.023111 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d59c77bf7-bcjl2" Jun 10 12:41:49.303351 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:49.302994 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:49.415973 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:49.415942 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:49.520059 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:49.520020 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:49.641208 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:49.641122 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:49.740598 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:49.740563 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:50.204474 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:50.204446 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:41:50.206112 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:50.206070 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:41:50.213317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:50.213296 2582 kubelet.go:1648] "Image garbage collection succeeded" Jun 10 12:41:51.626131 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.626078 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d858fb77f-ww6tq"] Jun 10 12:41:51.630909 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.630892 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.635599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.635343 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 10 12:41:51.635599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.635394 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jpzks\"" Jun 10 12:41:51.635599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.635417 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 10 12:41:51.635599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.635425 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 10 12:41:51.635599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.635425 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 10 12:41:51.635599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.635422 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 10 12:41:51.644226 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.644160 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d858fb77f-ww6tq"] Jun 10 12:41:51.754281 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.754247 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-oauth-serving-cert\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.754500 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.754303 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-service-ca\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.754500 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.754346 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpw4t\" (UniqueName: \"kubernetes.io/projected/95d82e30-8359-4e7a-92c6-698e8576d9d5-kube-api-access-bpw4t\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.754500 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.754399 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-serving-cert\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.754668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.754494 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-oauth-config\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.754668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.754528 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-config\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.855082 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.855048 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-oauth-serving-cert\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.855277 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.855091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-service-ca\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.855277 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.855141 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpw4t\" (UniqueName: \"kubernetes.io/projected/95d82e30-8359-4e7a-92c6-698e8576d9d5-kube-api-access-bpw4t\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.855277 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.855176 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-serving-cert\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.855277 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.855221 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-oauth-config\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.855277 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.855242 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-config\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.855905 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.855871 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-oauth-serving-cert\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.856033 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.855910 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-config\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.856033 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.855981 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-service-ca\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.858574 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.858550 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-serving-cert\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.858686 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.858554 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-oauth-config\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.868036 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.867980 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpw4t\" (UniqueName: \"kubernetes.io/projected/95d82e30-8359-4e7a-92c6-698e8576d9d5-kube-api-access-bpw4t\") pod \"console-d858fb77f-ww6tq\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:51.943249 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:51.943161 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:41:52.735037 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:52.735003 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:52.735480 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:52.735065 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:41:55.085181 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:55.085135 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:55.175114 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:55.175078 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:55.767146 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:55.767092 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:55.864585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:55.864550 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:56.043663 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:56.043589 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:56.193806 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:56.193752 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:56.489792 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:56.489747 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:56.608172 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:56.608141 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:57.505242 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:57.505207 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1da94685-3c98-4177-8917-027f02be0785/init-config-reloader/0.log" Jun 10 12:41:57.717277 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:57.717201 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1da94685-3c98-4177-8917-027f02be0785/alertmanager/0.log" Jun 10 12:41:57.911940 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:57.911910 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1da94685-3c98-4177-8917-027f02be0785/config-reloader/0.log" Jun 10 12:41:58.112695 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:58.112662 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1da94685-3c98-4177-8917-027f02be0785/kube-rbac-proxy-web/0.log" Jun 10 12:41:58.241941 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:58.241857 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:41:58.305188 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:58.305153 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1da94685-3c98-4177-8917-027f02be0785/kube-rbac-proxy/0.log" Jun 10 12:41:58.510031 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:58.509958 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1da94685-3c98-4177-8917-027f02be0785/kube-rbac-proxy-metric/0.log" Jun 10 12:41:58.706881 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:58.706859 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d858fb77f-ww6tq"] Jun 10 12:41:58.711123 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:58.711083 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1da94685-3c98-4177-8917-027f02be0785/prom-label-proxy/0.log" Jun 10 12:41:58.714820 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:58.714801 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:58.782429 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:58.782331 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:58.787641 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:41:58.787618 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d82e30_8359_4e7a_92c6_698e8576d9d5.slice/crio-fadc0f119b71816813722a6f0d6274fc5af2777382ad3a70d3c43fe2b526aa5f WatchSource:0}: Error finding container fadc0f119b71816813722a6f0d6274fc5af2777382ad3a70d3c43fe2b526aa5f: Status 404 returned error can't find the container with id fadc0f119b71816813722a6f0d6274fc5af2777382ad3a70d3c43fe2b526aa5f Jun 10 12:41:58.807773 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:58.807751 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 12:41:59.107255 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.107223 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-pd7jx_2d630daa-7080-4ae1-b427-0a901405386a/kube-state-metrics/0.log" Jun 10 12:41:59.215862 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.215824 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d858fb77f-ww6tq" event={"ID":"95d82e30-8359-4e7a-92c6-698e8576d9d5","Type":"ContainerStarted","Data":"fadc0f119b71816813722a6f0d6274fc5af2777382ad3a70d3c43fe2b526aa5f"} Jun 10 12:41:59.217759 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.217731 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-f8995" event={"ID":"066f8f06-a2e4-4703-9e42-ce8067800a2c","Type":"ContainerStarted","Data":"d197b7690deab797df8f3d28ac74c2876d1a6dc45a37ce99a81c2151126d5632"} Jun 10 12:41:59.218261 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.218214 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-f8995" Jun 10 12:41:59.226791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.226767 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-f8995" Jun 10 12:41:59.244279 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.244225 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-f8995" podStartSLOduration=1.576777684 podStartE2EDuration="18.244177885s" totalImagesPullingTime="16.667400201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:41:59.242424425 +0000 UTC m=+309.453270414" watchObservedRunningTime="2026-06-10 12:41:59.244177885 +0000 UTC m=+309.455023873" Jun 10 12:41:59.306597 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.306566 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-pd7jx_2d630daa-7080-4ae1-b427-0a901405386a/kube-rbac-proxy-main/0.log" Jun 10 12:41:59.480355 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.480270 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9b8f84446-m8gvr"] Jun 10 12:41:59.482943 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.482918 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.494322 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.494289 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 10 12:41:59.505397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.505366 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9b8f84446-m8gvr"] Jun 10 12:41:59.509431 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.509409 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-pd7jx_2d630daa-7080-4ae1-b427-0a901405386a/kube-rbac-proxy-self/0.log" Jun 10 12:41:59.629796 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.629749 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-serving-cert\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.630293 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.629818 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-config\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.630293 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.629849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-oauth-config\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.630293 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.629881 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-oauth-serving-cert\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.630293 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.629965 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-trusted-ca-bundle\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.630293 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.630004 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wvzb\" (UniqueName: \"kubernetes.io/projected/1425a895-acb7-4bd1-a26a-3b1d38c09903-kube-api-access-8wvzb\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.630293 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.630089 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-service-ca\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.703647 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.703616 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-867d7ffb85-s2wvx_dc283677-d4ee-4fab-a89f-e09a25a8cef6/metrics-server/0.log" Jun 10 12:41:59.730888 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.730787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-serving-cert\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.730888 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.730863 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-config\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.731151 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.730894 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-oauth-config\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.731151 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.730921 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-oauth-serving-cert\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.731151 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.731031 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-trusted-ca-bundle\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.731151 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.731120 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wvzb\" (UniqueName: \"kubernetes.io/projected/1425a895-acb7-4bd1-a26a-3b1d38c09903-kube-api-access-8wvzb\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.731151 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.731152 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-service-ca\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.731793 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.731713 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-oauth-serving-cert\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.731907 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.731802 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-service-ca\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.732582 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.732552 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-config\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.733288 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.733262 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-trusted-ca-bundle\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.735134 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.735073 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-oauth-config\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.736307 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.736214 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-serving-cert\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.743874 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.743763 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wvzb\" (UniqueName: \"kubernetes.io/projected/1425a895-acb7-4bd1-a26a-3b1d38c09903-kube-api-access-8wvzb\") pod \"console-9b8f84446-m8gvr\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.798708 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.798666 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:41:59.898518 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.898390 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:41:59.906266 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:41:59.906189 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-p6mrp_9f6621df-fea0-4d8b-95a9-b48be65a38cd/monitoring-plugin/0.log" Jun 10 12:42:00.005658 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:00.005569 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9b8f84446-m8gvr"] Jun 10 12:42:00.011659 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:42:00.011616 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1425a895_acb7_4bd1_a26a_3b1d38c09903.slice/crio-9b464bcfc309cbc2feeafbed6147a8aab442b5ec712fb5adc8898a6facdc64d6 WatchSource:0}: Error finding container 9b464bcfc309cbc2feeafbed6147a8aab442b5ec712fb5adc8898a6facdc64d6: Status 404 returned error can't find the container with id 9b464bcfc309cbc2feeafbed6147a8aab442b5ec712fb5adc8898a6facdc64d6 Jun 10 12:42:00.045687 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:00.045628 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:00.104057 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:00.103959 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjz24_ccb06a3a-3837-4662-bbfc-a2b80b648493/init-textfile/0.log" Jun 10 12:42:00.224470 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:00.224361 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9b8f84446-m8gvr" event={"ID":"1425a895-acb7-4bd1-a26a-3b1d38c09903","Type":"ContainerStarted","Data":"9b464bcfc309cbc2feeafbed6147a8aab442b5ec712fb5adc8898a6facdc64d6"} Jun 10 12:42:00.304262 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:00.304226 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjz24_ccb06a3a-3837-4662-bbfc-a2b80b648493/node-exporter/0.log" Jun 10 12:42:00.503394 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:00.503332 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjz24_ccb06a3a-3837-4662-bbfc-a2b80b648493/kube-rbac-proxy/0.log" Jun 10 12:42:04.247942 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:04.247903 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d858fb77f-ww6tq" event={"ID":"95d82e30-8359-4e7a-92c6-698e8576d9d5","Type":"ContainerStarted","Data":"135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e"} Jun 10 12:42:04.248814 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:04.248788 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9b8f84446-m8gvr" event={"ID":"1425a895-acb7-4bd1-a26a-3b1d38c09903","Type":"ContainerStarted","Data":"3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026"} Jun 10 12:42:04.274119 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:04.274043 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d858fb77f-ww6tq" podStartSLOduration=8.867694621 podStartE2EDuration="13.274026455s" totalImagesPullingTime="4.406331834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:41:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:42:04.273230162 +0000 UTC m=+314.484076150" watchObservedRunningTime="2026-06-10 12:42:04.274026455 +0000 UTC m=+314.484872442" Jun 10 12:42:04.315034 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:04.311044 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9b8f84446-m8gvr" podStartSLOduration=2.104850449 podStartE2EDuration="5.311022806s" totalImagesPullingTime="3.206172357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:42:04.309166134 +0000 UTC m=+314.520012122" watchObservedRunningTime="2026-06-10 12:42:04.311022806 +0000 UTC m=+314.521868793" Jun 10 12:42:05.670256 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:05.670206 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:05.772212 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:05.772153 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:06.304463 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:06.304435 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-mx9nj_3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0/networking-console-plugin/0.log" Jun 10 12:42:06.904366 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:06.904341 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d858fb77f-ww6tq_95d82e30-8359-4e7a-92c6-698e8576d9d5/console/0.log" Jun 10 12:42:07.107333 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:07.107298 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-f8995_066f8f06-a2e4-4703-9e42-ce8067800a2c/download-server/0.log" Jun 10 12:42:08.104686 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:08.104649 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmlx9_306d7497-fa10-42e3-8c0a-7fb50bf91206/dns/0.log" Jun 10 12:42:08.302999 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:08.302969 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmlx9_306d7497-fa10-42e3-8c0a-7fb50bf91206/kube-rbac-proxy/0.log" Jun 10 12:42:08.503595 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:08.503528 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5m4pd_666e8e25-9dde-4db2-a3e2-3e75b5e24059/dns-node-resolver/0.log" Jun 10 12:42:09.273470 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:09.273429 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:09.327220 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:09.327190 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:09.799209 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:09.799179 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:42:09.799460 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:09.799232 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:42:09.804349 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:09.804313 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:42:10.277264 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:10.277229 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:42:10.345119 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:10.345060 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d858fb77f-ww6tq"] Jun 10 12:42:10.470820 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:10.470795 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:10.571597 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:10.571568 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:10.719930 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:10.719897 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:10.789717 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:10.789683 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:11.140075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:11.140039 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:11.211982 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:11.211889 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:11.944312 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:11.944281 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:42:12.740736 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:12.740707 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:42:12.745430 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:12.745404 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-867d7ffb85-s2wvx" Jun 10 12:42:25.799001 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:25.798966 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:25.948726 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:25.948694 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:35.378809 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.378762 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-d858fb77f-ww6tq" podUID="95d82e30-8359-4e7a-92c6-698e8576d9d5" containerName="console" containerID="cri-o://135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e" gracePeriod=15 Jun 10 12:42:35.658784 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.658706 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:42:35.763979 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.763938 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-oauth-serving-cert\") pod \"95d82e30-8359-4e7a-92c6-698e8576d9d5\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " Jun 10 12:42:35.763979 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.763981 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpw4t\" (UniqueName: \"kubernetes.io/projected/95d82e30-8359-4e7a-92c6-698e8576d9d5-kube-api-access-bpw4t\") pod \"95d82e30-8359-4e7a-92c6-698e8576d9d5\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " Jun 10 12:42:35.764219 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.763997 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-serving-cert\") pod \"95d82e30-8359-4e7a-92c6-698e8576d9d5\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " Jun 10 12:42:35.764219 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.764047 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-service-ca\") pod \"95d82e30-8359-4e7a-92c6-698e8576d9d5\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " Jun 10 12:42:35.764219 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.764079 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-oauth-config\") pod \"95d82e30-8359-4e7a-92c6-698e8576d9d5\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " Jun 10 12:42:35.764219 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.764138 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-config\") pod \"95d82e30-8359-4e7a-92c6-698e8576d9d5\" (UID: \"95d82e30-8359-4e7a-92c6-698e8576d9d5\") " Jun 10 12:42:35.764443 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.764399 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "95d82e30-8359-4e7a-92c6-698e8576d9d5" (UID: "95d82e30-8359-4e7a-92c6-698e8576d9d5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:42:35.764548 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.764512 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-service-ca" (OuterVolumeSpecName: "service-ca") pod "95d82e30-8359-4e7a-92c6-698e8576d9d5" (UID: "95d82e30-8359-4e7a-92c6-698e8576d9d5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:42:35.764624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.764594 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-config" (OuterVolumeSpecName: "console-config") pod "95d82e30-8359-4e7a-92c6-698e8576d9d5" (UID: "95d82e30-8359-4e7a-92c6-698e8576d9d5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:42:35.766439 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.766409 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "95d82e30-8359-4e7a-92c6-698e8576d9d5" (UID: "95d82e30-8359-4e7a-92c6-698e8576d9d5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:42:35.766785 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.766768 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "95d82e30-8359-4e7a-92c6-698e8576d9d5" (UID: "95d82e30-8359-4e7a-92c6-698e8576d9d5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:42:35.766853 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.766793 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95d82e30-8359-4e7a-92c6-698e8576d9d5-kube-api-access-bpw4t" (OuterVolumeSpecName: "kube-api-access-bpw4t") pod "95d82e30-8359-4e7a-92c6-698e8576d9d5" (UID: "95d82e30-8359-4e7a-92c6-698e8576d9d5"). InnerVolumeSpecName "kube-api-access-bpw4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:42:35.864933 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.864882 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-oauth-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:35.864933 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.864921 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpw4t\" (UniqueName: \"kubernetes.io/projected/95d82e30-8359-4e7a-92c6-698e8576d9d5-kube-api-access-bpw4t\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:35.864933 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.864935 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:35.865212 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.864946 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:35.865212 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.864958 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-oauth-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:35.865212 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:35.864969 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/95d82e30-8359-4e7a-92c6-698e8576d9d5-console-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:36.370917 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:36.370882 2582 generic.go:356] "Generic (PLEG): container finished" podID="95d82e30-8359-4e7a-92c6-698e8576d9d5" containerID="135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e" exitCode=0 Jun 10 12:42:36.371084 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:36.370926 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d858fb77f-ww6tq" Jun 10 12:42:36.371084 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:36.370929 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d858fb77f-ww6tq" event={"ID":"95d82e30-8359-4e7a-92c6-698e8576d9d5","Type":"ContainerDied","Data":"135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e"} Jun 10 12:42:36.371084 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:36.371056 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d858fb77f-ww6tq" event={"ID":"95d82e30-8359-4e7a-92c6-698e8576d9d5","Type":"ContainerDied","Data":"fadc0f119b71816813722a6f0d6274fc5af2777382ad3a70d3c43fe2b526aa5f"} Jun 10 12:42:36.371084 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:36.371081 2582 scope.go:117] "RemoveContainer" containerID="135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e" Jun 10 12:42:36.379838 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:36.379744 2582 scope.go:117] "RemoveContainer" containerID="135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e" Jun 10 12:42:36.380127 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:42:36.380076 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e\": container with ID starting with 135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e not found: ID does not exist" containerID="135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e" Jun 10 12:42:36.380178 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:36.380117 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e"} err="failed to get container status \"135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e\": rpc error: code = NotFound desc = could not find container \"135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e\": container with ID starting with 135b94b398475c72b1cdd3b83fce684dbf7fb9fb5f67f6e852f29eab1df0902e not found: ID does not exist" Jun 10 12:42:36.395991 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:36.395963 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d858fb77f-ww6tq"] Jun 10 12:42:36.400149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:36.400126 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d858fb77f-ww6tq"] Jun 10 12:42:38.247205 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:38.247173 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95d82e30-8359-4e7a-92c6-698e8576d9d5" path="/var/lib/kubelet/pods/95d82e30-8359-4e7a-92c6-698e8576d9d5/volumes" Jun 10 12:42:40.747269 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:40.747236 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:40.805312 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:40.805280 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:44.215310 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:44.215276 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:44.278550 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:44.278504 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:48.368487 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.368453 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7756944cf8-cvd5q"] Jun 10 12:42:48.368885 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.368773 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d82e30-8359-4e7a-92c6-698e8576d9d5" containerName="console" Jun 10 12:42:48.368885 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.368784 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d82e30-8359-4e7a-92c6-698e8576d9d5" containerName="console" Jun 10 12:42:48.368885 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.368837 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="95d82e30-8359-4e7a-92c6-698e8576d9d5" containerName="console" Jun 10 12:42:48.371479 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.371457 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.382902 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.382878 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7756944cf8-cvd5q"] Jun 10 12:42:48.425534 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.425487 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:48.470571 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.470532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-service-ca\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.470726 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.470603 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-console-config\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.470726 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.470647 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-oauth-serving-cert\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.470726 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.470669 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-serving-cert\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.470726 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.470691 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bxkz\" (UniqueName: \"kubernetes.io/projected/ec616f84-5301-4afa-a2c9-df4154367be8-kube-api-access-7bxkz\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.470726 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.470726 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-trusted-ca-bundle\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.470962 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.470748 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-oauth-config\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.526652 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.526616 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:48.571639 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.571602 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-console-config\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.571831 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.571650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-oauth-serving-cert\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.571831 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.571710 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-serving-cert\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.571831 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.571730 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bxkz\" (UniqueName: \"kubernetes.io/projected/ec616f84-5301-4afa-a2c9-df4154367be8-kube-api-access-7bxkz\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.571831 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.571814 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-trusted-ca-bundle\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.572053 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.571872 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-oauth-config\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.572053 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.571978 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-service-ca\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.574545 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.573127 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-console-config\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.574545 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.573337 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-oauth-serving-cert\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.574545 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.573348 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-service-ca\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.574545 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.573716 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-trusted-ca-bundle\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.577778 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.576699 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-oauth-config\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.578218 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.578193 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-serving-cert\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.580439 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.580417 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:42:48.581147 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.581079 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="alertmanager" containerID="cri-o://caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891" gracePeriod=120 Jun 10 12:42:48.581321 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.581270 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy" containerID="cri-o://1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743" gracePeriod=120 Jun 10 12:42:48.581419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.581396 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="config-reloader" containerID="cri-o://9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3" gracePeriod=120 Jun 10 12:42:48.581511 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.581459 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="prom-label-proxy" containerID="cri-o://38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79" gracePeriod=120 Jun 10 12:42:48.581569 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.581466 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy-metric" containerID="cri-o://30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b" gracePeriod=120 Jun 10 12:42:48.581569 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.581495 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy-web" containerID="cri-o://56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1" gracePeriod=120 Jun 10 12:42:48.583505 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.583462 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bxkz\" (UniqueName: \"kubernetes.io/projected/ec616f84-5301-4afa-a2c9-df4154367be8-kube-api-access-7bxkz\") pod \"console-7756944cf8-cvd5q\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.681219 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.681119 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:48.820899 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:48.820830 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7756944cf8-cvd5q"] Jun 10 12:42:48.824290 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:42:48.824251 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec616f84_5301_4afa_a2c9_df4154367be8.slice/crio-db7f4952e3ae8d88b22aed957ef44a38df68b42abfd934f322f81710ef2d55f4 WatchSource:0}: Error finding container db7f4952e3ae8d88b22aed957ef44a38df68b42abfd934f322f81710ef2d55f4: Status 404 returned error can't find the container with id db7f4952e3ae8d88b22aed957ef44a38df68b42abfd934f322f81710ef2d55f4 Jun 10 12:42:49.422568 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.422534 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7756944cf8-cvd5q" event={"ID":"ec616f84-5301-4afa-a2c9-df4154367be8","Type":"ContainerStarted","Data":"3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6"} Jun 10 12:42:49.423038 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.422590 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7756944cf8-cvd5q" event={"ID":"ec616f84-5301-4afa-a2c9-df4154367be8","Type":"ContainerStarted","Data":"db7f4952e3ae8d88b22aed957ef44a38df68b42abfd934f322f81710ef2d55f4"} Jun 10 12:42:49.425971 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.425945 2582 generic.go:356] "Generic (PLEG): container finished" podID="1da94685-3c98-4177-8917-027f02be0785" containerID="38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79" exitCode=0 Jun 10 12:42:49.425971 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.425965 2582 generic.go:356] "Generic (PLEG): container finished" podID="1da94685-3c98-4177-8917-027f02be0785" containerID="30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b" exitCode=0 Jun 10 12:42:49.425971 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.425973 2582 generic.go:356] "Generic (PLEG): container finished" podID="1da94685-3c98-4177-8917-027f02be0785" containerID="1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743" exitCode=0 Jun 10 12:42:49.425971 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.425981 2582 generic.go:356] "Generic (PLEG): container finished" podID="1da94685-3c98-4177-8917-027f02be0785" containerID="9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3" exitCode=0 Jun 10 12:42:49.425971 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.425987 2582 generic.go:356] "Generic (PLEG): container finished" podID="1da94685-3c98-4177-8917-027f02be0785" containerID="caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891" exitCode=0 Jun 10 12:42:49.426241 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.426012 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerDied","Data":"38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79"} Jun 10 12:42:49.426241 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.426066 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerDied","Data":"30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b"} Jun 10 12:42:49.426241 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.426092 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerDied","Data":"1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743"} Jun 10 12:42:49.426241 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.426137 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerDied","Data":"9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3"} Jun 10 12:42:49.426241 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.426162 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerDied","Data":"caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891"} Jun 10 12:42:49.442439 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.442394 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7756944cf8-cvd5q" podStartSLOduration=1.4423809969999999 podStartE2EDuration="1.442380997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:42:49.441833174 +0000 UTC m=+359.652679163" watchObservedRunningTime="2026-06-10 12:42:49.442380997 +0000 UTC m=+359.653226984" Jun 10 12:42:49.819164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.819144 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:49.882687 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882657 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.882872 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882714 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-alertmanager-main-db\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.882872 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882754 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-tls-assets\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.882872 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882780 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-main-tls\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.882872 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882806 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.882872 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882833 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zb5p\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-kube-api-access-9zb5p\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.882872 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882868 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-config-out\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.883215 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882906 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-web\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.883215 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882960 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-web-config\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.883215 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.882987 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-config-volume\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.883215 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.883028 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-cluster-tls-config\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.883215 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.883068 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-metrics-client-ca\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.883215 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.883114 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-alertmanager-trusted-ca-bundle\") pod \"1da94685-3c98-4177-8917-027f02be0785\" (UID: \"1da94685-3c98-4177-8917-027f02be0785\") " Jun 10 12:42:49.883994 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.883963 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:42:49.884845 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.884543 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:42:49.884845 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.884635 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:42:49.886539 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.886514 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:42:49.886665 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.886591 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-config-volume" (OuterVolumeSpecName: "config-volume") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:42:49.886829 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.886802 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:42:49.887296 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.887262 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:42:49.892051 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.887842 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:42:49.892051 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.887959 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-config-out" (OuterVolumeSpecName: "config-out") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:42:49.892051 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.888411 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-kube-api-access-9zb5p" (OuterVolumeSpecName: "kube-api-access-9zb5p") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "kube-api-access-9zb5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:42:49.892051 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.888568 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:42:49.894850 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.894817 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:42:49.901976 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.901949 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-web-config" (OuterVolumeSpecName: "web-config") pod "1da94685-3c98-4177-8917-027f02be0785" (UID: "1da94685-3c98-4177-8917-027f02be0785"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:42:49.984585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984495 2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-metrics-client-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984531 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1da94685-3c98-4177-8917-027f02be0785-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984541 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984550 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-alertmanager-main-db\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984561 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-tls-assets\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984569 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-main-tls\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984577 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984586 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zb5p\" (UniqueName: \"kubernetes.io/projected/1da94685-3c98-4177-8917-027f02be0785-kube-api-access-9zb5p\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984594 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1da94685-3c98-4177-8917-027f02be0785-config-out\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984947 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984603 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984947 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984611 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-web-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984947 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984619 2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-config-volume\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:49.984947 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:49.984626 2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1da94685-3c98-4177-8917-027f02be0785-cluster-tls-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:42:50.432196 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.432158 2582 generic.go:356] "Generic (PLEG): container finished" podID="1da94685-3c98-4177-8917-027f02be0785" containerID="56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1" exitCode=0 Jun 10 12:42:50.432598 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.432242 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.432598 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.432246 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerDied","Data":"56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1"} Jun 10 12:42:50.432598 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.432328 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1da94685-3c98-4177-8917-027f02be0785","Type":"ContainerDied","Data":"2d87f748ac08b4daa7a20ac31be44c9fe4a00ff00e399f177d91909b7ac06024"} Jun 10 12:42:50.432598 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.432377 2582 scope.go:117] "RemoveContainer" containerID="38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79" Jun 10 12:42:50.440058 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.440041 2582 scope.go:117] "RemoveContainer" containerID="30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b" Jun 10 12:42:50.447558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.447534 2582 scope.go:117] "RemoveContainer" containerID="1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743" Jun 10 12:42:50.454826 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.454801 2582 scope.go:117] "RemoveContainer" containerID="56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1" Jun 10 12:42:50.455673 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.455653 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:42:50.462154 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.462072 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:42:50.463258 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.463238 2582 scope.go:117] "RemoveContainer" containerID="9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3" Jun 10 12:42:50.470501 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.470484 2582 scope.go:117] "RemoveContainer" containerID="caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891" Jun 10 12:42:50.477812 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.477793 2582 scope.go:117] "RemoveContainer" containerID="d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b" Jun 10 12:42:50.485380 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.485359 2582 scope.go:117] "RemoveContainer" containerID="38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79" Jun 10 12:42:50.485652 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:42:50.485631 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79\": container with ID starting with 38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79 not found: ID does not exist" containerID="38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79" Jun 10 12:42:50.485732 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.485664 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79"} err="failed to get container status \"38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79\": rpc error: code = NotFound desc = could not find container \"38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79\": container with ID starting with 38bf8d24bc277032472a209d1044746698467d565cea4c1ea83db23b7f69ac79 not found: ID does not exist" Jun 10 12:42:50.485732 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.485693 2582 scope.go:117] "RemoveContainer" containerID="30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b" Jun 10 12:42:50.485965 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:42:50.485944 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b\": container with ID starting with 30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b not found: ID does not exist" containerID="30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b" Jun 10 12:42:50.486041 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.485973 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b"} err="failed to get container status \"30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b\": rpc error: code = NotFound desc = could not find container \"30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b\": container with ID starting with 30d413ab8ca872125df4a9e29be92709936efdc39e8f1ca303ca93987a95f63b not found: ID does not exist" Jun 10 12:42:50.486041 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.485996 2582 scope.go:117] "RemoveContainer" containerID="1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743" Jun 10 12:42:50.486299 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:42:50.486261 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743\": container with ID starting with 1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743 not found: ID does not exist" containerID="1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743" Jun 10 12:42:50.486412 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.486291 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743"} err="failed to get container status \"1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743\": rpc error: code = NotFound desc = could not find container \"1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743\": container with ID starting with 1366bb81baa34adf86e68ae7a8271f663a4e17472146569934b0da97e0300743 not found: ID does not exist" Jun 10 12:42:50.486412 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.486312 2582 scope.go:117] "RemoveContainer" containerID="56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1" Jun 10 12:42:50.486548 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:42:50.486530 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1\": container with ID starting with 56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1 not found: ID does not exist" containerID="56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1" Jun 10 12:42:50.486610 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.486551 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1"} err="failed to get container status \"56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1\": rpc error: code = NotFound desc = could not find container \"56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1\": container with ID starting with 56d3a388d8f0eb399e1376dce111c27dae9691d8099cc12d4c2719014a94ffd1 not found: ID does not exist" Jun 10 12:42:50.486610 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.486569 2582 scope.go:117] "RemoveContainer" containerID="9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3" Jun 10 12:42:50.486888 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:42:50.486829 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3\": container with ID starting with 9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3 not found: ID does not exist" containerID="9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3" Jun 10 12:42:50.486888 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.486856 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3"} err="failed to get container status \"9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3\": rpc error: code = NotFound desc = could not find container \"9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3\": container with ID starting with 9ea9318073865f23795455948cae69444777356c60bfbd1221c34d51a6b1a1a3 not found: ID does not exist" Jun 10 12:42:50.486888 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.486875 2582 scope.go:117] "RemoveContainer" containerID="caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891" Jun 10 12:42:50.487152 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:42:50.487088 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891\": container with ID starting with caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891 not found: ID does not exist" containerID="caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891" Jun 10 12:42:50.487240 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.487155 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891"} err="failed to get container status \"caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891\": rpc error: code = NotFound desc = could not find container \"caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891\": container with ID starting with caaaa277dfff3197de8ff87fb85f8c88a53ea66490c703a83b8f612841888891 not found: ID does not exist" Jun 10 12:42:50.487240 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.487167 2582 scope.go:117] "RemoveContainer" containerID="d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b" Jun 10 12:42:50.487415 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:42:50.487397 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b\": container with ID starting with d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b not found: ID does not exist" containerID="d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b" Jun 10 12:42:50.487460 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.487422 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b"} err="failed to get container status \"d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b\": rpc error: code = NotFound desc = could not find container \"d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b\": container with ID starting with d3a34e90f8847620f139cc57d9b5f628cdf239a4761611e5c89f14f8b12d704b not found: ID does not exist" Jun 10 12:42:50.497063 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497036 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:42:50.497490 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497472 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="alertmanager" Jun 10 12:42:50.497541 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497494 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="alertmanager" Jun 10 12:42:50.497541 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497509 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="init-config-reloader" Jun 10 12:42:50.497541 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497518 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="init-config-reloader" Jun 10 12:42:50.497541 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497535 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="config-reloader" Jun 10 12:42:50.497668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497544 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="config-reloader" Jun 10 12:42:50.497668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497557 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy-web" Jun 10 12:42:50.497668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497565 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy-web" Jun 10 12:42:50.497668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497578 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy" Jun 10 12:42:50.497668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497587 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy" Jun 10 12:42:50.497668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497595 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="prom-label-proxy" Jun 10 12:42:50.497668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497603 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="prom-label-proxy" Jun 10 12:42:50.497668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497616 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy-metric" Jun 10 12:42:50.497668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497624 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy-metric" Jun 10 12:42:50.497912 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497688 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="prom-label-proxy" Jun 10 12:42:50.497912 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497696 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="config-reloader" Jun 10 12:42:50.497912 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497702 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy" Jun 10 12:42:50.497912 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497710 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="alertmanager" Jun 10 12:42:50.497912 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497714 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy-web" Jun 10 12:42:50.497912 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.497719 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1da94685-3c98-4177-8917-027f02be0785" containerName="kube-rbac-proxy-metric" Jun 10 12:42:50.501191 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.501173 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.504114 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.504079 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 10 12:42:50.504221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.504143 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-shmlb\"" Jun 10 12:42:50.504221 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.504178 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 10 12:42:50.504481 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.504433 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 10 12:42:50.504481 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.504457 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 10 12:42:50.504629 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.504521 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 10 12:42:50.504733 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.504690 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 10 12:42:50.504733 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.504709 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 10 12:42:50.504866 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.504844 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 10 12:42:50.511337 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.511318 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 10 12:42:50.521603 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.521583 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:42:50.589252 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589208 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3df88ada-e928-4047-ad3a-2a01f762add1-config-out\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589258 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-web-config\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589287 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589304 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589327 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3df88ada-e928-4047-ad3a-2a01f762add1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589362 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589595 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-config-volume\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589595 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589432 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3df88ada-e928-4047-ad3a-2a01f762add1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589595 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589475 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz2gp\" (UniqueName: \"kubernetes.io/projected/3df88ada-e928-4047-ad3a-2a01f762add1-kube-api-access-qz2gp\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589595 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589502 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3df88ada-e928-4047-ad3a-2a01f762add1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589595 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589521 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589595 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589561 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.589595 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.589594 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3df88ada-e928-4047-ad3a-2a01f762add1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.690815 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.690776 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.690815 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.690817 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.690838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3df88ada-e928-4047-ad3a-2a01f762add1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.690866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3df88ada-e928-4047-ad3a-2a01f762add1-config-out\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.690884 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-web-config\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.690903 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.690920 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.690940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3df88ada-e928-4047-ad3a-2a01f762add1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.690987 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.691015 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-config-volume\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691039 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.691037 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3df88ada-e928-4047-ad3a-2a01f762add1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.691073 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz2gp\" (UniqueName: \"kubernetes.io/projected/3df88ada-e928-4047-ad3a-2a01f762add1-kube-api-access-qz2gp\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.691149 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3df88ada-e928-4047-ad3a-2a01f762add1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.691543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.691316 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3df88ada-e928-4047-ad3a-2a01f762add1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.694181 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.694153 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3df88ada-e928-4047-ad3a-2a01f762add1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.694301 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.694196 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.694382 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.694327 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3df88ada-e928-4047-ad3a-2a01f762add1-config-out\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.694509 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.694481 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.694624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.694580 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-web-config\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.695289 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.694789 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.695289 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.695043 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.695289 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.695076 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.695289 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.695247 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3df88ada-e928-4047-ad3a-2a01f762add1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.695531 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.695299 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3df88ada-e928-4047-ad3a-2a01f762add1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.696522 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.696502 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3df88ada-e928-4047-ad3a-2a01f762add1-config-volume\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.704273 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.704252 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz2gp\" (UniqueName: \"kubernetes.io/projected/3df88ada-e928-4047-ad3a-2a01f762add1-kube-api-access-qz2gp\") pod \"alertmanager-main-0\" (UID: \"3df88ada-e928-4047-ad3a-2a01f762add1\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.810752 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.810725 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:42:50.952069 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:50.952037 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:42:50.954429 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:42:50.954405 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df88ada_e928_4047_ad3a_2a01f762add1.slice/crio-a95eea0c29bc9b22730ce014cfe9849c6a1a9e69307f3ce8b4024d476de8890c WatchSource:0}: Error finding container a95eea0c29bc9b22730ce014cfe9849c6a1a9e69307f3ce8b4024d476de8890c: Status 404 returned error can't find the container with id a95eea0c29bc9b22730ce014cfe9849c6a1a9e69307f3ce8b4024d476de8890c Jun 10 12:42:51.438004 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:51.437971 2582 generic.go:356] "Generic (PLEG): container finished" podID="3df88ada-e928-4047-ad3a-2a01f762add1" containerID="6e363cf48de5f65066c05c3802f36c49b904ab94efb1cccaa128cd9681db7a1e" exitCode=0 Jun 10 12:42:51.438413 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:51.438036 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3df88ada-e928-4047-ad3a-2a01f762add1","Type":"ContainerDied","Data":"6e363cf48de5f65066c05c3802f36c49b904ab94efb1cccaa128cd9681db7a1e"} Jun 10 12:42:51.438413 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:51.438086 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3df88ada-e928-4047-ad3a-2a01f762add1","Type":"ContainerStarted","Data":"a95eea0c29bc9b22730ce014cfe9849c6a1a9e69307f3ce8b4024d476de8890c"} Jun 10 12:42:52.246363 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:52.246317 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1da94685-3c98-4177-8917-027f02be0785" path="/var/lib/kubelet/pods/1da94685-3c98-4177-8917-027f02be0785/volumes" Jun 10 12:42:52.445366 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:52.445330 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3df88ada-e928-4047-ad3a-2a01f762add1","Type":"ContainerStarted","Data":"61ab1a30735c8887c4588f29097aef17f68e8877eee8f74e0d04fa7054bdfe9a"} Jun 10 12:42:52.445720 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:52.445402 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3df88ada-e928-4047-ad3a-2a01f762add1","Type":"ContainerStarted","Data":"9c5696d49b65f9fa90e2fff3999fa18d22f3e5a4d47bc80b57e0cb3e239f021a"} Jun 10 12:42:52.445720 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:52.445429 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3df88ada-e928-4047-ad3a-2a01f762add1","Type":"ContainerStarted","Data":"cf54021eb2886e454e1283d80df24aff30b70d205577a5cd1d8940e1d442b44d"} Jun 10 12:42:52.445720 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:52.445470 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3df88ada-e928-4047-ad3a-2a01f762add1","Type":"ContainerStarted","Data":"a1026ab5e9c2fb9b2828c6f01c852ac2c7eb486671b651fdd83afc56febbdde0"} Jun 10 12:42:52.445720 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:52.445499 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3df88ada-e928-4047-ad3a-2a01f762add1","Type":"ContainerStarted","Data":"4b6db7ba29d87ebdd90d76fc44ec597ba915e3123e33d553aed843d220f4f245"} Jun 10 12:42:52.445720 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:52.445555 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3df88ada-e928-4047-ad3a-2a01f762add1","Type":"ContainerStarted","Data":"6eb9f1e7e9bca5e3fcc8ff778cee41e9e30807fea737b2c43772a779a55ae826"} Jun 10 12:42:52.480134 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:52.480067 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.466612085 podStartE2EDuration="2.480051239s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.439154ms" isStatefulPod=false podCreationTimestamp="2026-06-10 12:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:42:52.477871088 +0000 UTC m=+362.688717076" watchObservedRunningTime="2026-06-10 12:42:52.480051239 +0000 UTC m=+362.690897203" Jun 10 12:42:58.294677 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:58.294622 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:58.394853 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:58.394799 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:58.682053 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:58.682015 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:58.682053 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:58.682063 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:58.686975 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:58.686950 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:59.478520 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:59.478493 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:42:59.568027 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:59.567995 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9b8f84446-m8gvr"] Jun 10 12:42:59.687622 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:59.687593 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:59.786047 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:59.785978 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:59.891684 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:59.891653 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:42:59.963157 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:42:59.963060 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:04.203484 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:04.203446 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:43:04.205960 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:04.205932 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433d9f02-513b-40d5-b647-9a9a3b4fa4e8-metrics-certs\") pod \"network-metrics-daemon-zlgzd\" (UID: \"433d9f02-513b-40d5-b647-9a9a3b4fa4e8\") " pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:43:04.246117 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:04.246073 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vpjkq\"" Jun 10 12:43:04.253698 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:04.253664 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zlgzd" Jun 10 12:43:04.385616 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:04.385372 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zlgzd"] Jun 10 12:43:04.391677 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:43:04.391644 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433d9f02_513b_40d5_b647_9a9a3b4fa4e8.slice/crio-96fec8a77aeaad8a0b4a408868fa4b82f189a83cc80ad14d637e30ff2030f418 WatchSource:0}: Error finding container 96fec8a77aeaad8a0b4a408868fa4b82f189a83cc80ad14d637e30ff2030f418: Status 404 returned error can't find the container with id 96fec8a77aeaad8a0b4a408868fa4b82f189a83cc80ad14d637e30ff2030f418 Jun 10 12:43:04.496132 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:04.496030 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zlgzd" event={"ID":"433d9f02-513b-40d5-b647-9a9a3b4fa4e8","Type":"ContainerStarted","Data":"96fec8a77aeaad8a0b4a408868fa4b82f189a83cc80ad14d637e30ff2030f418"} Jun 10 12:43:06.506118 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:06.506059 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zlgzd" event={"ID":"433d9f02-513b-40d5-b647-9a9a3b4fa4e8","Type":"ContainerStarted","Data":"9df22c7fc430e672088d6cf19ac31251e28c33576ab343f9b0a96b6542d9b7da"} Jun 10 12:43:06.506477 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:06.506126 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zlgzd" event={"ID":"433d9f02-513b-40d5-b647-9a9a3b4fa4e8","Type":"ContainerStarted","Data":"129a6fd03b9f664010f12ad53b91599b23e790ad41af9d8064bad8c8cd1219e9"} Jun 10 12:43:06.539023 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:06.538965 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zlgzd" podStartSLOduration=375.219542686 podStartE2EDuration="6m16.538948095s" totalImagesPullingTime="1.319405409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:43:06.538539611 +0000 UTC m=+376.749385597" watchObservedRunningTime="2026-06-10 12:43:06.538948095 +0000 UTC m=+376.749794083" Jun 10 12:43:06.662050 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:06.662016 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:06.737292 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:06.737259 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:08.484249 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:08.484216 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:08.569901 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:08.569867 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:10.762647 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:10.762611 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:10.860815 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:10.860779 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:22.169086 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:22.169054 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:24.588192 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.588137 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-9b8f84446-m8gvr" podUID="1425a895-acb7-4bd1-a26a-3b1d38c09903" containerName="console" containerID="cri-o://3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026" gracePeriod=15 Jun 10 12:43:24.851477 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.851408 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:43:24.986041 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986011 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-oauth-config\") pod \"1425a895-acb7-4bd1-a26a-3b1d38c09903\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " Jun 10 12:43:24.986041 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986043 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wvzb\" (UniqueName: \"kubernetes.io/projected/1425a895-acb7-4bd1-a26a-3b1d38c09903-kube-api-access-8wvzb\") pod \"1425a895-acb7-4bd1-a26a-3b1d38c09903\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " Jun 10 12:43:24.986298 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986063 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-trusted-ca-bundle\") pod \"1425a895-acb7-4bd1-a26a-3b1d38c09903\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " Jun 10 12:43:24.986298 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986120 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-config\") pod \"1425a895-acb7-4bd1-a26a-3b1d38c09903\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " Jun 10 12:43:24.986298 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986259 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-service-ca\") pod \"1425a895-acb7-4bd1-a26a-3b1d38c09903\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " Jun 10 12:43:24.986448 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986358 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-serving-cert\") pod \"1425a895-acb7-4bd1-a26a-3b1d38c09903\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " Jun 10 12:43:24.986448 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986442 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-oauth-serving-cert\") pod \"1425a895-acb7-4bd1-a26a-3b1d38c09903\" (UID: \"1425a895-acb7-4bd1-a26a-3b1d38c09903\") " Jun 10 12:43:24.986579 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986551 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1425a895-acb7-4bd1-a26a-3b1d38c09903" (UID: "1425a895-acb7-4bd1-a26a-3b1d38c09903"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:43:24.986625 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986575 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-config" (OuterVolumeSpecName: "console-config") pod "1425a895-acb7-4bd1-a26a-3b1d38c09903" (UID: "1425a895-acb7-4bd1-a26a-3b1d38c09903"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:43:24.986723 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986684 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-service-ca" (OuterVolumeSpecName: "service-ca") pod "1425a895-acb7-4bd1-a26a-3b1d38c09903" (UID: "1425a895-acb7-4bd1-a26a-3b1d38c09903"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:43:24.986723 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986709 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-trusted-ca-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:24.986855 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.986741 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:24.987322 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.987296 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1425a895-acb7-4bd1-a26a-3b1d38c09903" (UID: "1425a895-acb7-4bd1-a26a-3b1d38c09903"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:43:24.988467 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.988443 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1425a895-acb7-4bd1-a26a-3b1d38c09903-kube-api-access-8wvzb" (OuterVolumeSpecName: "kube-api-access-8wvzb") pod "1425a895-acb7-4bd1-a26a-3b1d38c09903" (UID: "1425a895-acb7-4bd1-a26a-3b1d38c09903"). InnerVolumeSpecName "kube-api-access-8wvzb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:43:24.988557 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.988476 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1425a895-acb7-4bd1-a26a-3b1d38c09903" (UID: "1425a895-acb7-4bd1-a26a-3b1d38c09903"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:43:24.988557 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:24.988494 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1425a895-acb7-4bd1-a26a-3b1d38c09903" (UID: "1425a895-acb7-4bd1-a26a-3b1d38c09903"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:43:25.087727 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.087673 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:25.087727 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.087725 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:25.087727 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.087736 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1425a895-acb7-4bd1-a26a-3b1d38c09903-oauth-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:25.087727 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.087746 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1425a895-acb7-4bd1-a26a-3b1d38c09903-console-oauth-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:25.087996 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.087754 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wvzb\" (UniqueName: \"kubernetes.io/projected/1425a895-acb7-4bd1-a26a-3b1d38c09903-kube-api-access-8wvzb\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:25.583483 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.583443 2582 generic.go:356] "Generic (PLEG): container finished" podID="1425a895-acb7-4bd1-a26a-3b1d38c09903" containerID="3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026" exitCode=0 Jun 10 12:43:25.583653 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.583489 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9b8f84446-m8gvr" Jun 10 12:43:25.583653 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.583518 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9b8f84446-m8gvr" event={"ID":"1425a895-acb7-4bd1-a26a-3b1d38c09903","Type":"ContainerDied","Data":"3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026"} Jun 10 12:43:25.583653 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.583568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9b8f84446-m8gvr" event={"ID":"1425a895-acb7-4bd1-a26a-3b1d38c09903","Type":"ContainerDied","Data":"9b464bcfc309cbc2feeafbed6147a8aab442b5ec712fb5adc8898a6facdc64d6"} Jun 10 12:43:25.583653 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.583588 2582 scope.go:117] "RemoveContainer" containerID="3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026" Jun 10 12:43:25.592397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.592211 2582 scope.go:117] "RemoveContainer" containerID="3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026" Jun 10 12:43:25.592624 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:43:25.592476 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026\": container with ID starting with 3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026 not found: ID does not exist" containerID="3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026" Jun 10 12:43:25.592624 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.592499 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026"} err="failed to get container status \"3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026\": rpc error: code = NotFound desc = could not find container \"3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026\": container with ID starting with 3f1a2442faade6d52d79c2ac27718f6e220fbb713fafbb4a4d5190bd4b945026 not found: ID does not exist" Jun 10 12:43:25.606497 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.606464 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9b8f84446-m8gvr"] Jun 10 12:43:25.610379 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:25.610352 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9b8f84446-m8gvr"] Jun 10 12:43:26.247733 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:26.247697 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1425a895-acb7-4bd1-a26a-3b1d38c09903" path="/var/lib/kubelet/pods/1425a895-acb7-4bd1-a26a-3b1d38c09903/volumes" Jun 10 12:43:27.194755 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:27.194712 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:27.296142 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:27.296091 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:30.482050 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:30.482013 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:30.583887 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:30.583854 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:31.497438 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.497392 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4"] Jun 10 12:43:31.497832 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.497704 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1425a895-acb7-4bd1-a26a-3b1d38c09903" containerName="console" Jun 10 12:43:31.497832 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.497715 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1425a895-acb7-4bd1-a26a-3b1d38c09903" containerName="console" Jun 10 12:43:31.497832 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.497766 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1425a895-acb7-4bd1-a26a-3b1d38c09903" containerName="console" Jun 10 12:43:31.500723 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.500705 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.503854 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.503830 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:43:31.503854 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.503855 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhnkq\"" Jun 10 12:43:31.505160 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.505138 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:43:31.517453 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.517423 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4"] Jun 10 12:43:31.642491 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.642450 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.642686 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.642499 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l6fr\" (UniqueName: \"kubernetes.io/projected/69925d66-7c9a-4c75-9bec-1747911ba277-kube-api-access-5l6fr\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.642686 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.642568 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.743946 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.743909 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l6fr\" (UniqueName: \"kubernetes.io/projected/69925d66-7c9a-4c75-9bec-1747911ba277-kube-api-access-5l6fr\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.744154 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.743955 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.744154 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.744009 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.744375 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.744356 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.744452 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.744405 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.757644 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.757581 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l6fr\" (UniqueName: \"kubernetes.io/projected/69925d66-7c9a-4c75-9bec-1747911ba277-kube-api-access-5l6fr\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.809621 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.809571 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:31.952994 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:31.952942 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4"] Jun 10 12:43:31.955837 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:43:31.955801 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69925d66_7c9a_4c75_9bec_1747911ba277.slice/crio-1e30d2db1bb4ef097bbdc97ccad23c07e5e91973b0e8c54330d000b6262bd282 WatchSource:0}: Error finding container 1e30d2db1bb4ef097bbdc97ccad23c07e5e91973b0e8c54330d000b6262bd282: Status 404 returned error can't find the container with id 1e30d2db1bb4ef097bbdc97ccad23c07e5e91973b0e8c54330d000b6262bd282 Jun 10 12:43:32.613870 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:32.613837 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" event={"ID":"69925d66-7c9a-4c75-9bec-1747911ba277","Type":"ContainerStarted","Data":"1e30d2db1bb4ef097bbdc97ccad23c07e5e91973b0e8c54330d000b6262bd282"} Jun 10 12:43:38.641451 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:38.641412 2582 generic.go:356] "Generic (PLEG): container finished" podID="69925d66-7c9a-4c75-9bec-1747911ba277" containerID="919d1d77760316c906c343e1e10c901ecd310b3c82d262380d58cd99cac1d88e" exitCode=0 Jun 10 12:43:38.641797 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:38.641505 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" event={"ID":"69925d66-7c9a-4c75-9bec-1747911ba277","Type":"ContainerDied","Data":"919d1d77760316c906c343e1e10c901ecd310b3c82d262380d58cd99cac1d88e"} Jun 10 12:43:41.654459 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:41.654421 2582 generic.go:356] "Generic (PLEG): container finished" podID="69925d66-7c9a-4c75-9bec-1747911ba277" containerID="516f504db5aa6dd08fc28ae7cbb3c8235b9131aa26f468b7e3b2a5f3c231c454" exitCode=0 Jun 10 12:43:41.654863 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:41.654488 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" event={"ID":"69925d66-7c9a-4c75-9bec-1747911ba277","Type":"ContainerDied","Data":"516f504db5aa6dd08fc28ae7cbb3c8235b9131aa26f468b7e3b2a5f3c231c454"} Jun 10 12:43:50.697312 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:50.697272 2582 generic.go:356] "Generic (PLEG): container finished" podID="69925d66-7c9a-4c75-9bec-1747911ba277" containerID="0ab717f98a82ec4f9fc0a30d0e7a8b9c405d72cc4e2f15149a9afb51ce213932" exitCode=0 Jun 10 12:43:50.697741 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:50.697353 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" event={"ID":"69925d66-7c9a-4c75-9bec-1747911ba277","Type":"ContainerDied","Data":"0ab717f98a82ec4f9fc0a30d0e7a8b9c405d72cc4e2f15149a9afb51ce213932"} Jun 10 12:43:51.880281 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:51.880243 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:51.913844 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:51.913810 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-util\") pod \"69925d66-7c9a-4c75-9bec-1747911ba277\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " Jun 10 12:43:51.914150 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:51.914124 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-bundle\") pod \"69925d66-7c9a-4c75-9bec-1747911ba277\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " Jun 10 12:43:51.914269 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:51.914237 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l6fr\" (UniqueName: \"kubernetes.io/projected/69925d66-7c9a-4c75-9bec-1747911ba277-kube-api-access-5l6fr\") pod \"69925d66-7c9a-4c75-9bec-1747911ba277\" (UID: \"69925d66-7c9a-4c75-9bec-1747911ba277\") " Jun 10 12:43:51.914783 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:51.914756 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-bundle" (OuterVolumeSpecName: "bundle") pod "69925d66-7c9a-4c75-9bec-1747911ba277" (UID: "69925d66-7c9a-4c75-9bec-1747911ba277"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:43:51.917112 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:51.917073 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69925d66-7c9a-4c75-9bec-1747911ba277-kube-api-access-5l6fr" (OuterVolumeSpecName: "kube-api-access-5l6fr") pod "69925d66-7c9a-4c75-9bec-1747911ba277" (UID: "69925d66-7c9a-4c75-9bec-1747911ba277"). InnerVolumeSpecName "kube-api-access-5l6fr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:43:51.918898 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:51.918877 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-util" (OuterVolumeSpecName: "util") pod "69925d66-7c9a-4c75-9bec-1747911ba277" (UID: "69925d66-7c9a-4c75-9bec-1747911ba277"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:43:52.015348 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:52.015259 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:52.015348 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:52.015289 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69925d66-7c9a-4c75-9bec-1747911ba277-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:52.015348 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:52.015298 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l6fr\" (UniqueName: \"kubernetes.io/projected/69925d66-7c9a-4c75-9bec-1747911ba277-kube-api-access-5l6fr\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:43:52.706943 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:52.706914 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" Jun 10 12:43:52.707138 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:52.706933 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dt9zt4" event={"ID":"69925d66-7c9a-4c75-9bec-1747911ba277","Type":"ContainerDied","Data":"1e30d2db1bb4ef097bbdc97ccad23c07e5e91973b0e8c54330d000b6262bd282"} Jun 10 12:43:52.707138 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:52.706989 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e30d2db1bb4ef097bbdc97ccad23c07e5e91973b0e8c54330d000b6262bd282" Jun 10 12:43:57.866175 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:57.866136 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:58.678933 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:58.678889 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:43:59.999899 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:43:59.999814 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl"] Jun 10 12:44:00.000308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.000120 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69925d66-7c9a-4c75-9bec-1747911ba277" containerName="extract" Jun 10 12:44:00.000308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.000131 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="69925d66-7c9a-4c75-9bec-1747911ba277" containerName="extract" Jun 10 12:44:00.000308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.000151 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69925d66-7c9a-4c75-9bec-1747911ba277" containerName="util" Jun 10 12:44:00.000308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.000156 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="69925d66-7c9a-4c75-9bec-1747911ba277" containerName="util" Jun 10 12:44:00.000308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.000162 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69925d66-7c9a-4c75-9bec-1747911ba277" containerName="pull" Jun 10 12:44:00.000308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.000168 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="69925d66-7c9a-4c75-9bec-1747911ba277" containerName="pull" Jun 10 12:44:00.000308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.000244 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="69925d66-7c9a-4c75-9bec-1747911ba277" containerName="extract" Jun 10 12:44:00.003040 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.003013 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" Jun 10 12:44:00.005949 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.005929 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qphr9\"" Jun 10 12:44:00.006353 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.006327 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 10 12:44:00.006353 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.006344 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:44:00.021599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.021567 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl"] Jun 10 12:44:00.084179 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.084146 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/401b15cf-b39a-4e07-a5c4-eb23263d388b-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-thjxl\" (UID: \"401b15cf-b39a-4e07-a5c4-eb23263d388b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" Jun 10 12:44:00.084395 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.084233 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg6cg\" (UniqueName: \"kubernetes.io/projected/401b15cf-b39a-4e07-a5c4-eb23263d388b-kube-api-access-cg6cg\") pod \"cert-manager-operator-controller-manager-57757f7cfc-thjxl\" (UID: \"401b15cf-b39a-4e07-a5c4-eb23263d388b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" Jun 10 12:44:00.184891 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.184839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg6cg\" (UniqueName: \"kubernetes.io/projected/401b15cf-b39a-4e07-a5c4-eb23263d388b-kube-api-access-cg6cg\") pod \"cert-manager-operator-controller-manager-57757f7cfc-thjxl\" (UID: \"401b15cf-b39a-4e07-a5c4-eb23263d388b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" Jun 10 12:44:00.185084 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.184907 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/401b15cf-b39a-4e07-a5c4-eb23263d388b-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-thjxl\" (UID: \"401b15cf-b39a-4e07-a5c4-eb23263d388b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" Jun 10 12:44:00.185317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.185296 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/401b15cf-b39a-4e07-a5c4-eb23263d388b-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-thjxl\" (UID: \"401b15cf-b39a-4e07-a5c4-eb23263d388b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" Jun 10 12:44:00.196887 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.196852 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg6cg\" (UniqueName: \"kubernetes.io/projected/401b15cf-b39a-4e07-a5c4-eb23263d388b-kube-api-access-cg6cg\") pod \"cert-manager-operator-controller-manager-57757f7cfc-thjxl\" (UID: \"401b15cf-b39a-4e07-a5c4-eb23263d388b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" Jun 10 12:44:00.311936 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.311909 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" Jun 10 12:44:00.455301 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.455274 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl"] Jun 10 12:44:00.458057 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:44:00.458029 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod401b15cf_b39a_4e07_a5c4_eb23263d388b.slice/crio-7c850ae0c6049a0fda72c9f38f845924125d0d5702d50888389e4dec4f643b03 WatchSource:0}: Error finding container 7c850ae0c6049a0fda72c9f38f845924125d0d5702d50888389e4dec4f643b03: Status 404 returned error can't find the container with id 7c850ae0c6049a0fda72c9f38f845924125d0d5702d50888389e4dec4f643b03 Jun 10 12:44:00.738938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:00.738846 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" event={"ID":"401b15cf-b39a-4e07-a5c4-eb23263d388b","Type":"ContainerStarted","Data":"7c850ae0c6049a0fda72c9f38f845924125d0d5702d50888389e4dec4f643b03"} Jun 10 12:44:01.839688 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:01.839649 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:01.940225 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:01.940183 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:02.749646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:02.749604 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" event={"ID":"401b15cf-b39a-4e07-a5c4-eb23263d388b","Type":"ContainerStarted","Data":"7bba5d4b48227dbb0a8b95175f043b29bafd2d1ec9d84e3ccdae652d8ab272f9"} Jun 10 12:44:02.788442 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:02.788306 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-thjxl" podStartSLOduration=2.042648405 podStartE2EDuration="3.788286194s" totalImagesPullingTime="1.745637789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:44:02.785932931 +0000 UTC m=+432.996778920" watchObservedRunningTime="2026-06-10 12:44:02.788286194 +0000 UTC m=+432.999132186" Jun 10 12:44:03.016474 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:03.016398 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:03.180289 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:03.180255 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:03.345604 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:03.345560 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:04.282721 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.282685 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh"] Jun 10 12:44:04.286341 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.286320 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.289300 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.289279 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhnkq\"" Jun 10 12:44:04.289515 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.289499 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:44:04.290493 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.290479 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:44:04.305396 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.305368 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh"] Jun 10 12:44:04.319239 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.319212 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.319386 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.319262 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r99zk\" (UniqueName: \"kubernetes.io/projected/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-kube-api-access-r99zk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.319386 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.319319 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.420703 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.420664 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.420875 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.420711 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r99zk\" (UniqueName: \"kubernetes.io/projected/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-kube-api-access-r99zk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.420875 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.420761 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.421071 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.421052 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.421136 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.421076 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.441564 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.441533 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r99zk\" (UniqueName: \"kubernetes.io/projected/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-kube-api-access-r99zk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.595917 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.595880 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:04.731925 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.731896 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh"] Jun 10 12:44:04.736823 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:44:04.736789 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02cf3cdd_bbf9_47a0_8b9c_dd3e2c78af37.slice/crio-84802deb29dbdc05b238f64445bc6ca1a01494a75f1af99a08e087490348a2dc WatchSource:0}: Error finding container 84802deb29dbdc05b238f64445bc6ca1a01494a75f1af99a08e087490348a2dc: Status 404 returned error can't find the container with id 84802deb29dbdc05b238f64445bc6ca1a01494a75f1af99a08e087490348a2dc Jun 10 12:44:04.760044 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:04.760012 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" event={"ID":"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37","Type":"ContainerStarted","Data":"84802deb29dbdc05b238f64445bc6ca1a01494a75f1af99a08e087490348a2dc"} Jun 10 12:44:05.765877 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:05.765785 2582 generic.go:356] "Generic (PLEG): container finished" podID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerID="5ded4e1236dd6abeeb760deac85c91bc4f3fbc9583d3da10b0b245d3408397f9" exitCode=0 Jun 10 12:44:05.765877 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:05.765860 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" event={"ID":"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37","Type":"ContainerDied","Data":"5ded4e1236dd6abeeb760deac85c91bc4f3fbc9583d3da10b0b245d3408397f9"} Jun 10 12:44:06.571055 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:06.571024 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:06.868509 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:06.868431 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:08.513161 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:08.513060 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:08.691875 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:08.691839 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:08.781216 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:08.781124 2582 generic.go:356] "Generic (PLEG): container finished" podID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerID="e2791944d2c703effa0477f5965224c9008e46e3e9ed6f6346ccab1ee5aae1be" exitCode=0 Jun 10 12:44:08.781358 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:08.781212 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" event={"ID":"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37","Type":"ContainerDied","Data":"e2791944d2c703effa0477f5965224c9008e46e3e9ed6f6346ccab1ee5aae1be"} Jun 10 12:44:09.788621 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:09.788588 2582 generic.go:356] "Generic (PLEG): container finished" podID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerID="c513ddcc4a11f88030ec617d4bd94b6622d87b5da39c9809f3905e73cf31b8b8" exitCode=0 Jun 10 12:44:09.789079 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:09.788654 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" event={"ID":"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37","Type":"ContainerDied","Data":"c513ddcc4a11f88030ec617d4bd94b6622d87b5da39c9809f3905e73cf31b8b8"} Jun 10 12:44:10.921152 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:10.921119 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:10.980230 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:10.980192 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r99zk\" (UniqueName: \"kubernetes.io/projected/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-kube-api-access-r99zk\") pod \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " Jun 10 12:44:10.980230 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:10.980235 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-bundle\") pod \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " Jun 10 12:44:10.980454 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:10.980275 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-util\") pod \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\" (UID: \"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37\") " Jun 10 12:44:10.980671 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:10.980646 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-bundle" (OuterVolumeSpecName: "bundle") pod "02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" (UID: "02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:44:10.982586 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:10.982553 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-kube-api-access-r99zk" (OuterVolumeSpecName: "kube-api-access-r99zk") pod "02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" (UID: "02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37"). InnerVolumeSpecName "kube-api-access-r99zk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:44:10.984768 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:10.984746 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-util" (OuterVolumeSpecName: "util") pod "02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" (UID: "02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:44:11.081478 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:11.081440 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r99zk\" (UniqueName: \"kubernetes.io/projected/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-kube-api-access-r99zk\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:44:11.081478 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:11.081471 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:44:11.081478 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:11.081480 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:44:11.799349 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:11.799317 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" event={"ID":"02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37","Type":"ContainerDied","Data":"84802deb29dbdc05b238f64445bc6ca1a01494a75f1af99a08e087490348a2dc"} Jun 10 12:44:11.799496 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:11.799364 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84802deb29dbdc05b238f64445bc6ca1a01494a75f1af99a08e087490348a2dc" Jun 10 12:44:11.799496 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:11.799327 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6bsbh" Jun 10 12:44:13.527824 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:13.527781 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:13.666446 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:13.666409 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:13.756530 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:13.756495 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:15.503423 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.503352 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:15.603999 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.603965 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:15.607767 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.607742 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-6sxrr"] Jun 10 12:44:15.608054 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.608042 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerName="pull" Jun 10 12:44:15.608116 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.608055 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerName="pull" Jun 10 12:44:15.608116 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.608062 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerName="extract" Jun 10 12:44:15.608116 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.608068 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerName="extract" Jun 10 12:44:15.608116 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.608074 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerName="util" Jun 10 12:44:15.608116 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.608080 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerName="util" Jun 10 12:44:15.608289 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.608159 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="02cf3cdd-bbf9-47a0-8b9c-dd3e2c78af37" containerName="extract" Jun 10 12:44:15.612394 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.612377 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-6sxrr" Jun 10 12:44:15.615080 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.615055 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 10 12:44:15.615303 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.615290 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 10 12:44:15.616419 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.616402 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hch2x\"" Jun 10 12:44:15.622886 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.622863 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-6sxrr"] Jun 10 12:44:15.719078 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.719024 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc6wj\" (UniqueName: \"kubernetes.io/projected/17fd8266-b4b8-4715-927a-be31c9505089-kube-api-access-mc6wj\") pod \"cert-manager-59c77684cc-6sxrr\" (UID: \"17fd8266-b4b8-4715-927a-be31c9505089\") " pod="cert-manager/cert-manager-59c77684cc-6sxrr" Jun 10 12:44:15.719288 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.719128 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17fd8266-b4b8-4715-927a-be31c9505089-bound-sa-token\") pod \"cert-manager-59c77684cc-6sxrr\" (UID: \"17fd8266-b4b8-4715-927a-be31c9505089\") " pod="cert-manager/cert-manager-59c77684cc-6sxrr" Jun 10 12:44:15.820321 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.820291 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc6wj\" (UniqueName: \"kubernetes.io/projected/17fd8266-b4b8-4715-927a-be31c9505089-kube-api-access-mc6wj\") pod \"cert-manager-59c77684cc-6sxrr\" (UID: \"17fd8266-b4b8-4715-927a-be31c9505089\") " pod="cert-manager/cert-manager-59c77684cc-6sxrr" Jun 10 12:44:15.820499 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.820329 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17fd8266-b4b8-4715-927a-be31c9505089-bound-sa-token\") pod \"cert-manager-59c77684cc-6sxrr\" (UID: \"17fd8266-b4b8-4715-927a-be31c9505089\") " pod="cert-manager/cert-manager-59c77684cc-6sxrr" Jun 10 12:44:15.832344 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.832320 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17fd8266-b4b8-4715-927a-be31c9505089-bound-sa-token\") pod \"cert-manager-59c77684cc-6sxrr\" (UID: \"17fd8266-b4b8-4715-927a-be31c9505089\") " pod="cert-manager/cert-manager-59c77684cc-6sxrr" Jun 10 12:44:15.832448 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.832377 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc6wj\" (UniqueName: \"kubernetes.io/projected/17fd8266-b4b8-4715-927a-be31c9505089-kube-api-access-mc6wj\") pod \"cert-manager-59c77684cc-6sxrr\" (UID: \"17fd8266-b4b8-4715-927a-be31c9505089\") " pod="cert-manager/cert-manager-59c77684cc-6sxrr" Jun 10 12:44:15.937042 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:15.937006 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-6sxrr" Jun 10 12:44:16.072390 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:16.072363 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-6sxrr"] Jun 10 12:44:16.074719 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:44:16.074695 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17fd8266_b4b8_4715_927a_be31c9505089.slice/crio-47dfe2803f3acb965909fe8e7f76f3556b7620b5204afbb6bbed962cf4971065 WatchSource:0}: Error finding container 47dfe2803f3acb965909fe8e7f76f3556b7620b5204afbb6bbed962cf4971065: Status 404 returned error can't find the container with id 47dfe2803f3acb965909fe8e7f76f3556b7620b5204afbb6bbed962cf4971065 Jun 10 12:44:16.824603 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:16.824569 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-6sxrr" event={"ID":"17fd8266-b4b8-4715-927a-be31c9505089","Type":"ContainerStarted","Data":"47dfe2803f3acb965909fe8e7f76f3556b7620b5204afbb6bbed962cf4971065"} Jun 10 12:44:16.880956 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:16.880921 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c"] Jun 10 12:44:16.886618 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:16.886600 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" Jun 10 12:44:16.889424 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:16.889403 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:44:16.889424 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:16.889412 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-r2b5w\"" Jun 10 12:44:16.890608 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:16.890594 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 10 12:44:16.897751 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:16.897729 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c"] Jun 10 12:44:17.034241 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.034198 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71ef3587-45e9-4489-8cf3-22eff70a970e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g2b2c\" (UID: \"71ef3587-45e9-4489-8cf3-22eff70a970e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" Jun 10 12:44:17.034436 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.034321 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pkfx\" (UniqueName: \"kubernetes.io/projected/71ef3587-45e9-4489-8cf3-22eff70a970e-kube-api-access-6pkfx\") pod \"openshift-lws-operator-fd8ccff4c-g2b2c\" (UID: \"71ef3587-45e9-4489-8cf3-22eff70a970e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" Jun 10 12:44:17.135558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.135479 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pkfx\" (UniqueName: \"kubernetes.io/projected/71ef3587-45e9-4489-8cf3-22eff70a970e-kube-api-access-6pkfx\") pod \"openshift-lws-operator-fd8ccff4c-g2b2c\" (UID: \"71ef3587-45e9-4489-8cf3-22eff70a970e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" Jun 10 12:44:17.135558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.135537 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71ef3587-45e9-4489-8cf3-22eff70a970e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g2b2c\" (UID: \"71ef3587-45e9-4489-8cf3-22eff70a970e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" Jun 10 12:44:17.135996 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.135973 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71ef3587-45e9-4489-8cf3-22eff70a970e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g2b2c\" (UID: \"71ef3587-45e9-4489-8cf3-22eff70a970e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" Jun 10 12:44:17.151771 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.151717 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pkfx\" (UniqueName: \"kubernetes.io/projected/71ef3587-45e9-4489-8cf3-22eff70a970e-kube-api-access-6pkfx\") pod \"openshift-lws-operator-fd8ccff4c-g2b2c\" (UID: \"71ef3587-45e9-4489-8cf3-22eff70a970e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" Jun 10 12:44:17.196328 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.196289 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" Jun 10 12:44:17.365553 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.365529 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c"] Jun 10 12:44:17.368788 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:44:17.368756 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71ef3587_45e9_4489_8cf3_22eff70a970e.slice/crio-9f053d02a28ed289effe23f50b46e5ab9db90243643af24e0de29dc47c55a03e WatchSource:0}: Error finding container 9f053d02a28ed289effe23f50b46e5ab9db90243643af24e0de29dc47c55a03e: Status 404 returned error can't find the container with id 9f053d02a28ed289effe23f50b46e5ab9db90243643af24e0de29dc47c55a03e Jun 10 12:44:17.834370 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.834335 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" event={"ID":"71ef3587-45e9-4489-8cf3-22eff70a970e","Type":"ContainerStarted","Data":"9f053d02a28ed289effe23f50b46e5ab9db90243643af24e0de29dc47c55a03e"} Jun 10 12:44:17.871312 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:17.871274 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:19.846025 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:19.845979 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-6sxrr" event={"ID":"17fd8266-b4b8-4715-927a-be31c9505089","Type":"ContainerStarted","Data":"a15abb3a03b9a30714defa2e866bf824bea41ef497600db14cc2ab8806404880"} Jun 10 12:44:19.870672 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:19.870624 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-6sxrr" podStartSLOduration=1.5620244859999999 podStartE2EDuration="4.870607363s" totalImagesPullingTime="3.308582877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:44:19.867991154 +0000 UTC m=+450.078837143" watchObservedRunningTime="2026-06-10 12:44:19.870607363 +0000 UTC m=+450.081453350" Jun 10 12:44:21.856795 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:21.856756 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" event={"ID":"71ef3587-45e9-4489-8cf3-22eff70a970e","Type":"ContainerStarted","Data":"7c1ad8e823db927390a96088286183bcd33c1da46c319a3d4cd833f3828d3d4d"} Jun 10 12:44:21.878294 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:21.878227 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g2b2c" podStartSLOduration=2.030142777 podStartE2EDuration="5.878204439s" totalImagesPullingTime="3.848061662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:44:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:44:21.87577286 +0000 UTC m=+452.086618849" watchObservedRunningTime="2026-06-10 12:44:21.878204439 +0000 UTC m=+452.089050427" Jun 10 12:44:22.610617 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:22.610577 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:22.710805 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:22.710772 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:24.306668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.306638 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:24.390319 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.390285 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p"] Jun 10 12:44:24.413492 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.413459 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p"] Jun 10 12:44:24.413651 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.413594 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.416710 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.416683 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:44:24.416858 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.416692 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:44:24.418151 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.418131 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhnkq\"" Jun 10 12:44:24.503896 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.503854 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bhcd\" (UniqueName: \"kubernetes.io/projected/76d89522-4159-4766-867b-07d1f2203ba2-kube-api-access-9bhcd\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.503896 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.503898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.504201 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.503966 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.605257 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.605153 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bhcd\" (UniqueName: \"kubernetes.io/projected/76d89522-4159-4766-867b-07d1f2203ba2-kube-api-access-9bhcd\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.605257 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.605194 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.605257 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.605233 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.605616 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.605596 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.605677 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.605616 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.615594 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.615565 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bhcd\" (UniqueName: \"kubernetes.io/projected/76d89522-4159-4766-867b-07d1f2203ba2-kube-api-access-9bhcd\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.723578 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.723539 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:24.867805 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:24.864012 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p"] Jun 10 12:44:25.878443 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:25.878350 2582 generic.go:356] "Generic (PLEG): container finished" podID="76d89522-4159-4766-867b-07d1f2203ba2" containerID="9bcf3a2622d901ecbd4cc667fca631596d64059c34e8a47c6ae62f7d6b7d2ad6" exitCode=0 Jun 10 12:44:25.878851 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:25.878433 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" event={"ID":"76d89522-4159-4766-867b-07d1f2203ba2","Type":"ContainerDied","Data":"9bcf3a2622d901ecbd4cc667fca631596d64059c34e8a47c6ae62f7d6b7d2ad6"} Jun 10 12:44:25.878851 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:25.878480 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" event={"ID":"76d89522-4159-4766-867b-07d1f2203ba2","Type":"ContainerStarted","Data":"371d08bf249786c897541ea289b96a003615ca8d8a42f0c3216452b811fc491b"} Jun 10 12:44:26.884632 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:26.884596 2582 generic.go:356] "Generic (PLEG): container finished" podID="76d89522-4159-4766-867b-07d1f2203ba2" containerID="172ac0d6f36ac712c2be6bea975931a49a628641b15da4c5f7d466278491c2af" exitCode=0 Jun 10 12:44:26.885023 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:26.884636 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" event={"ID":"76d89522-4159-4766-867b-07d1f2203ba2","Type":"ContainerDied","Data":"172ac0d6f36ac712c2be6bea975931a49a628641b15da4c5f7d466278491c2af"} Jun 10 12:44:27.891321 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:27.891290 2582 generic.go:356] "Generic (PLEG): container finished" podID="76d89522-4159-4766-867b-07d1f2203ba2" containerID="2ea893184a5066638bf60015347e400206f372177590cc2974c6ce7c7d1c6ae9" exitCode=0 Jun 10 12:44:27.891684 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:27.891401 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" event={"ID":"76d89522-4159-4766-867b-07d1f2203ba2","Type":"ContainerDied","Data":"2ea893184a5066638bf60015347e400206f372177590cc2974c6ce7c7d1c6ae9"} Jun 10 12:44:29.027741 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.027715 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:29.144220 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.144183 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bhcd\" (UniqueName: \"kubernetes.io/projected/76d89522-4159-4766-867b-07d1f2203ba2-kube-api-access-9bhcd\") pod \"76d89522-4159-4766-867b-07d1f2203ba2\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " Jun 10 12:44:29.144387 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.144313 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-bundle\") pod \"76d89522-4159-4766-867b-07d1f2203ba2\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " Jun 10 12:44:29.144387 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.144330 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-util\") pod \"76d89522-4159-4766-867b-07d1f2203ba2\" (UID: \"76d89522-4159-4766-867b-07d1f2203ba2\") " Jun 10 12:44:29.145058 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.145029 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-bundle" (OuterVolumeSpecName: "bundle") pod "76d89522-4159-4766-867b-07d1f2203ba2" (UID: "76d89522-4159-4766-867b-07d1f2203ba2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:44:29.146540 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.146489 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d89522-4159-4766-867b-07d1f2203ba2-kube-api-access-9bhcd" (OuterVolumeSpecName: "kube-api-access-9bhcd") pod "76d89522-4159-4766-867b-07d1f2203ba2" (UID: "76d89522-4159-4766-867b-07d1f2203ba2"). InnerVolumeSpecName "kube-api-access-9bhcd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:44:29.149575 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.149541 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-util" (OuterVolumeSpecName: "util") pod "76d89522-4159-4766-867b-07d1f2203ba2" (UID: "76d89522-4159-4766-867b-07d1f2203ba2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:44:29.244972 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.244936 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:44:29.244972 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.244965 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76d89522-4159-4766-867b-07d1f2203ba2-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:44:29.244972 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.244978 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bhcd\" (UniqueName: \"kubernetes.io/projected/76d89522-4159-4766-867b-07d1f2203ba2-kube-api-access-9bhcd\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:44:29.902240 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.902213 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" Jun 10 12:44:29.902240 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.902221 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdvzp4p" event={"ID":"76d89522-4159-4766-867b-07d1f2203ba2","Type":"ContainerDied","Data":"371d08bf249786c897541ea289b96a003615ca8d8a42f0c3216452b811fc491b"} Jun 10 12:44:29.902450 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:29.902261 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="371d08bf249786c897541ea289b96a003615ca8d8a42f0c3216452b811fc491b" Jun 10 12:44:36.073678 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:36.073643 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:36.180811 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:36.180773 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:36.674023 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:36.673979 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:37.026060 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.025984 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:37.123848 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.123821 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:37.184668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.184634 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7"] Jun 10 12:44:37.184976 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.184964 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d89522-4159-4766-867b-07d1f2203ba2" containerName="extract" Jun 10 12:44:37.185015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.184982 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d89522-4159-4766-867b-07d1f2203ba2" containerName="extract" Jun 10 12:44:37.185015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.184993 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d89522-4159-4766-867b-07d1f2203ba2" containerName="util" Jun 10 12:44:37.185015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.184999 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d89522-4159-4766-867b-07d1f2203ba2" containerName="util" Jun 10 12:44:37.185015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.185006 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d89522-4159-4766-867b-07d1f2203ba2" containerName="pull" Jun 10 12:44:37.185015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.185011 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d89522-4159-4766-867b-07d1f2203ba2" containerName="pull" Jun 10 12:44:37.185186 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.185063 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d89522-4159-4766-867b-07d1f2203ba2" containerName="extract" Jun 10 12:44:37.188130 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.188085 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.192128 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.192078 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 10 12:44:37.192128 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.192123 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s7ppv\"" Jun 10 12:44:37.192311 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.192166 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 10 12:44:37.192311 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.192267 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 10 12:44:37.193841 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.193823 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 10 12:44:37.220139 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.220082 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7"] Jun 10 12:44:37.314984 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.314945 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:37.316147 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.316123 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d281b21d-ebd1-4296-9694-f4fcb17d9805-manifests\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.316200 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.316163 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d281b21d-ebd1-4296-9694-f4fcb17d9805-tmp\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.316264 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.316247 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d281b21d-ebd1-4296-9694-f4fcb17d9805-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.316305 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.316284 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pch7d\" (UniqueName: \"kubernetes.io/projected/d281b21d-ebd1-4296-9694-f4fcb17d9805-kube-api-access-pch7d\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.316366 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.316352 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d281b21d-ebd1-4296-9694-f4fcb17d9805-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.415679 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.415646 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:37.416773 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.416754 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d281b21d-ebd1-4296-9694-f4fcb17d9805-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.416838 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.416785 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pch7d\" (UniqueName: \"kubernetes.io/projected/d281b21d-ebd1-4296-9694-f4fcb17d9805-kube-api-access-pch7d\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.416971 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.416948 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d281b21d-ebd1-4296-9694-f4fcb17d9805-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.417078 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.417062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d281b21d-ebd1-4296-9694-f4fcb17d9805-manifests\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.417155 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.417127 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d281b21d-ebd1-4296-9694-f4fcb17d9805-tmp\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.417485 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.417458 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d281b21d-ebd1-4296-9694-f4fcb17d9805-manifests\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.419417 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.419388 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d281b21d-ebd1-4296-9694-f4fcb17d9805-tmp\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.419532 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.419512 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d281b21d-ebd1-4296-9694-f4fcb17d9805-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.419572 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.419512 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d281b21d-ebd1-4296-9694-f4fcb17d9805-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.427981 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.427960 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pch7d\" (UniqueName: \"kubernetes.io/projected/d281b21d-ebd1-4296-9694-f4fcb17d9805-kube-api-access-pch7d\") pod \"opendatahub-operator-controller-manager-6c7bd9b77-cqgr7\" (UID: \"d281b21d-ebd1-4296-9694-f4fcb17d9805\") " pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.498535 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.498500 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:37.518042 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.518010 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:37.599455 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.599420 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:37.640125 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.640085 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7"] Jun 10 12:44:37.642233 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:44:37.642205 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd281b21d_ebd1_4296_9694_f4fcb17d9805.slice/crio-7c4f4a5e5d8d58d294b4252636cc1e938031f6a6bc290ec42c9b99551c2f0e4c WatchSource:0}: Error finding container 7c4f4a5e5d8d58d294b4252636cc1e938031f6a6bc290ec42c9b99551c2f0e4c: Status 404 returned error can't find the container with id 7c4f4a5e5d8d58d294b4252636cc1e938031f6a6bc290ec42c9b99551c2f0e4c Jun 10 12:44:37.675043 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.675014 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:37.797588 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.797547 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:37.945694 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:37.945594 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" event={"ID":"d281b21d-ebd1-4296-9694-f4fcb17d9805","Type":"ContainerStarted","Data":"7c4f4a5e5d8d58d294b4252636cc1e938031f6a6bc290ec42c9b99551c2f0e4c"} Jun 10 12:44:40.963044 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:40.963000 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" event={"ID":"d281b21d-ebd1-4296-9694-f4fcb17d9805","Type":"ContainerStarted","Data":"8a6838dbb1a916f10233bddde27b83e5db886ba654aa982634c524c00226556b"} Jun 10 12:44:41.976648 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:41.976550 2582 generic.go:356] "Generic (PLEG): container finished" podID="d281b21d-ebd1-4296-9694-f4fcb17d9805" containerID="8a6838dbb1a916f10233bddde27b83e5db886ba654aa982634c524c00226556b" exitCode=0 Jun 10 12:44:41.976648 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:41.976634 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" event={"ID":"d281b21d-ebd1-4296-9694-f4fcb17d9805","Type":"ContainerDied","Data":"8a6838dbb1a916f10233bddde27b83e5db886ba654aa982634c524c00226556b"} Jun 10 12:44:42.983538 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:42.983494 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" event={"ID":"d281b21d-ebd1-4296-9694-f4fcb17d9805","Type":"ContainerStarted","Data":"a8ff8baff3cf4ed51d0a6094878649ecb3bc68b619218aeb6ce5dc142d6b5666"} Jun 10 12:44:42.984002 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:42.983643 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:43.012702 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:43.012651 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" podStartSLOduration=1.818211035 podStartE2EDuration="6.012634451s" totalImagesPullingTime="3.194243143s" totalInitContainerRuntime="1.000180273s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:44:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:44:43.010361338 +0000 UTC m=+473.221207350" watchObservedRunningTime="2026-06-10 12:44:43.012634451 +0000 UTC m=+473.223480437" Jun 10 12:44:53.990381 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:53.990342 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c7bd9b77-cqgr7" Jun 10 12:44:55.818002 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:55.817968 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:56.055348 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:56.055310 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:56.767683 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:56.767656 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:56.964640 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:56.964609 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:57.528014 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.527970 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:57.702158 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.702122 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-zwbrs"] Jun 10 12:44:57.712300 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.712265 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7567468478-wz9pp"] Jun 10 12:44:57.712482 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.712420 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:44:57.715736 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.715707 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.715929 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.715885 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kczsn\"" Jun 10 12:44:57.716526 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.716502 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 10 12:44:57.721465 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.721159 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 10 12:44:57.721465 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.721269 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5ft6x\"" Jun 10 12:44:57.721465 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.721391 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 10 12:44:57.723622 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.723595 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-zwbrs"] Jun 10 12:44:57.741484 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.741452 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:57.749213 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.749178 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7567468478-wz9pp"] Jun 10 12:44:57.770015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.769978 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6"] Jun 10 12:44:57.773806 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.773778 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:57.777643 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.777618 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:44:57.777772 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.777702 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhnkq\"" Jun 10 12:44:57.777772 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.777709 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:44:57.792827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.792794 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6"] Jun 10 12:44:57.795543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.795223 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cxwf\" (UniqueName: \"kubernetes.io/projected/70700fe6-5cd4-4a4b-b646-ccc3c4eee484-kube-api-access-4cxwf\") pod \"kube-auth-proxy-7567468478-wz9pp\" (UID: \"70700fe6-5cd4-4a4b-b646-ccc3c4eee484\") " pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.795543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.795277 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqfnf\" (UniqueName: \"kubernetes.io/projected/4a81b239-4f98-42d6-8de2-c1e069633bc5-kube-api-access-sqfnf\") pod \"odh-model-controller-6dc94f689f-zwbrs\" (UID: \"4a81b239-4f98-42d6-8de2-c1e069633bc5\") " pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:44:57.795543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.795367 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a81b239-4f98-42d6-8de2-c1e069633bc5-cert\") pod \"odh-model-controller-6dc94f689f-zwbrs\" (UID: \"4a81b239-4f98-42d6-8de2-c1e069633bc5\") " pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:44:57.795543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.795392 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70700fe6-5cd4-4a4b-b646-ccc3c4eee484-tmp\") pod \"kube-auth-proxy-7567468478-wz9pp\" (UID: \"70700fe6-5cd4-4a4b-b646-ccc3c4eee484\") " pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.795543 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.795489 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70700fe6-5cd4-4a4b-b646-ccc3c4eee484-tls-certs\") pod \"kube-auth-proxy-7567468478-wz9pp\" (UID: \"70700fe6-5cd4-4a4b-b646-ccc3c4eee484\") " pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.853952 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.853920 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:57.896263 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.896229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqfnf\" (UniqueName: \"kubernetes.io/projected/4a81b239-4f98-42d6-8de2-c1e069633bc5-kube-api-access-sqfnf\") pod \"odh-model-controller-6dc94f689f-zwbrs\" (UID: \"4a81b239-4f98-42d6-8de2-c1e069633bc5\") " pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:44:57.896477 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.896271 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwr65\" (UniqueName: \"kubernetes.io/projected/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-kube-api-access-zwr65\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:57.896477 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.896296 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:57.896477 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.896334 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a81b239-4f98-42d6-8de2-c1e069633bc5-cert\") pod \"odh-model-controller-6dc94f689f-zwbrs\" (UID: \"4a81b239-4f98-42d6-8de2-c1e069633bc5\") " pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:44:57.896477 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.896366 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70700fe6-5cd4-4a4b-b646-ccc3c4eee484-tmp\") pod \"kube-auth-proxy-7567468478-wz9pp\" (UID: \"70700fe6-5cd4-4a4b-b646-ccc3c4eee484\") " pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.896477 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:44:57.896426 2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 10 12:44:57.896477 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.896447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70700fe6-5cd4-4a4b-b646-ccc3c4eee484-tls-certs\") pod \"kube-auth-proxy-7567468478-wz9pp\" (UID: \"70700fe6-5cd4-4a4b-b646-ccc3c4eee484\") " pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.896806 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:44:57.896484 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a81b239-4f98-42d6-8de2-c1e069633bc5-cert podName:4a81b239-4f98-42d6-8de2-c1e069633bc5 nodeName:}" failed. No retries permitted until 2026-06-10 12:44:58.396462429 +0000 UTC m=+488.607308394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4a81b239-4f98-42d6-8de2-c1e069633bc5-cert") pod "odh-model-controller-6dc94f689f-zwbrs" (UID: "4a81b239-4f98-42d6-8de2-c1e069633bc5") : secret "odh-model-controller-webhook-cert" not found Jun 10 12:44:57.896806 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.896525 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:57.896806 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.896553 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cxwf\" (UniqueName: \"kubernetes.io/projected/70700fe6-5cd4-4a4b-b646-ccc3c4eee484-kube-api-access-4cxwf\") pod \"kube-auth-proxy-7567468478-wz9pp\" (UID: \"70700fe6-5cd4-4a4b-b646-ccc3c4eee484\") " pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.898983 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.898957 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70700fe6-5cd4-4a4b-b646-ccc3c4eee484-tmp\") pod \"kube-auth-proxy-7567468478-wz9pp\" (UID: \"70700fe6-5cd4-4a4b-b646-ccc3c4eee484\") " pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.899130 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.898994 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70700fe6-5cd4-4a4b-b646-ccc3c4eee484-tls-certs\") pod \"kube-auth-proxy-7567468478-wz9pp\" (UID: \"70700fe6-5cd4-4a4b-b646-ccc3c4eee484\") " pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.928555 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.928520 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqfnf\" (UniqueName: \"kubernetes.io/projected/4a81b239-4f98-42d6-8de2-c1e069633bc5-kube-api-access-sqfnf\") pod \"odh-model-controller-6dc94f689f-zwbrs\" (UID: \"4a81b239-4f98-42d6-8de2-c1e069633bc5\") " pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:44:57.938211 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.938169 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cxwf\" (UniqueName: \"kubernetes.io/projected/70700fe6-5cd4-4a4b-b646-ccc3c4eee484-kube-api-access-4cxwf\") pod \"kube-auth-proxy-7567468478-wz9pp\" (UID: \"70700fe6-5cd4-4a4b-b646-ccc3c4eee484\") " pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:57.963434 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.963390 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:57.998013 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.997979 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:57.998499 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.998032 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwr65\" (UniqueName: \"kubernetes.io/projected/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-kube-api-access-zwr65\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:57.998499 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.998059 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:57.998499 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.998459 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:57.998499 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:57.998471 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:58.027429 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.027395 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwr65\" (UniqueName: \"kubernetes.io/projected/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-kube-api-access-zwr65\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:58.039696 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.039621 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" Jun 10 12:44:58.083615 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.083581 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:44:58.131448 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.131404 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:58.250263 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.250234 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7567468478-wz9pp"] Jun 10 12:44:58.250578 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:44:58.250558 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70700fe6_5cd4_4a4b_b646_ccc3c4eee484.slice/crio-cfc71ffeab9baf097189dc5b7f9349db5751d1a3382a675abd9b7520e2c821d6 WatchSource:0}: Error finding container cfc71ffeab9baf097189dc5b7f9349db5751d1a3382a675abd9b7520e2c821d6: Status 404 returned error can't find the container with id cfc71ffeab9baf097189dc5b7f9349db5751d1a3382a675abd9b7520e2c821d6 Jun 10 12:44:58.267718 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.267690 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6"] Jun 10 12:44:58.273332 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:44:58.273303 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192ad9c9_58ae_4738_a56d_f16f8cf3dec7.slice/crio-5f80165707be70c7fa8388b8ca04c7275980e8595724ad1aa05f5939c2118127 WatchSource:0}: Error finding container 5f80165707be70c7fa8388b8ca04c7275980e8595724ad1aa05f5939c2118127: Status 404 returned error can't find the container with id 5f80165707be70c7fa8388b8ca04c7275980e8595724ad1aa05f5939c2118127 Jun 10 12:44:58.284006 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.283984 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:58.402558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.402527 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a81b239-4f98-42d6-8de2-c1e069633bc5-cert\") pod \"odh-model-controller-6dc94f689f-zwbrs\" (UID: \"4a81b239-4f98-42d6-8de2-c1e069633bc5\") " pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:44:58.405044 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.405011 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a81b239-4f98-42d6-8de2-c1e069633bc5-cert\") pod \"odh-model-controller-6dc94f689f-zwbrs\" (UID: \"4a81b239-4f98-42d6-8de2-c1e069633bc5\") " pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:44:58.629040 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.628995 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:44:58.794393 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.794364 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-zwbrs"] Jun 10 12:44:58.796383 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:44:58.796353 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a81b239_4f98_42d6_8de2_c1e069633bc5.slice/crio-fd151e3ce8fb5552a654e5ccbeb5f791bc043b58dc79c385784fafb5286873bc WatchSource:0}: Error finding container fd151e3ce8fb5552a654e5ccbeb5f791bc043b58dc79c385784fafb5286873bc: Status 404 returned error can't find the container with id fd151e3ce8fb5552a654e5ccbeb5f791bc043b58dc79c385784fafb5286873bc Jun 10 12:44:58.805159 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.805137 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:44:58.867384 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.867349 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-cpfzp"] Jun 10 12:44:58.871894 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.871872 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:44:58.875344 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.875316 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 10 12:44:58.875480 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.875363 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-zqvs8\"" Jun 10 12:44:58.891015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:58.890945 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-cpfzp"] Jun 10 12:44:59.007172 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.007138 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fh57\" (UniqueName: \"kubernetes.io/projected/aa0cfac9-cacd-4e3d-954a-757de8e42280-kube-api-access-8fh57\") pod \"model-serving-api-5dd488fc48-cpfzp\" (UID: \"aa0cfac9-cacd-4e3d-954a-757de8e42280\") " pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:44:59.007679 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.007646 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa0cfac9-cacd-4e3d-954a-757de8e42280-tls-certs\") pod \"model-serving-api-5dd488fc48-cpfzp\" (UID: \"aa0cfac9-cacd-4e3d-954a-757de8e42280\") " pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:44:59.064066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.064025 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" event={"ID":"70700fe6-5cd4-4a4b-b646-ccc3c4eee484","Type":"ContainerStarted","Data":"cfc71ffeab9baf097189dc5b7f9349db5751d1a3382a675abd9b7520e2c821d6"} Jun 10 12:44:59.069421 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.069321 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" event={"ID":"4a81b239-4f98-42d6-8de2-c1e069633bc5","Type":"ContainerStarted","Data":"fd151e3ce8fb5552a654e5ccbeb5f791bc043b58dc79c385784fafb5286873bc"} Jun 10 12:44:59.072558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.072470 2582 generic.go:356] "Generic (PLEG): container finished" podID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerID="e657efc5ccff8da5875f1705926d67d9277bc6b633238e3f99803af9e06d9474" exitCode=0 Jun 10 12:44:59.072686 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.072561 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" event={"ID":"192ad9c9-58ae-4738-a56d-f16f8cf3dec7","Type":"ContainerDied","Data":"e657efc5ccff8da5875f1705926d67d9277bc6b633238e3f99803af9e06d9474"} Jun 10 12:44:59.075178 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.073564 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" event={"ID":"192ad9c9-58ae-4738-a56d-f16f8cf3dec7","Type":"ContainerStarted","Data":"5f80165707be70c7fa8388b8ca04c7275980e8595724ad1aa05f5939c2118127"} Jun 10 12:44:59.110036 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.109565 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fh57\" (UniqueName: \"kubernetes.io/projected/aa0cfac9-cacd-4e3d-954a-757de8e42280-kube-api-access-8fh57\") pod \"model-serving-api-5dd488fc48-cpfzp\" (UID: \"aa0cfac9-cacd-4e3d-954a-757de8e42280\") " pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:44:59.110036 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.109637 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa0cfac9-cacd-4e3d-954a-757de8e42280-tls-certs\") pod \"model-serving-api-5dd488fc48-cpfzp\" (UID: \"aa0cfac9-cacd-4e3d-954a-757de8e42280\") " pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:44:59.110036 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:44:59.109856 2582 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 10 12:44:59.110036 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:44:59.109923 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa0cfac9-cacd-4e3d-954a-757de8e42280-tls-certs podName:aa0cfac9-cacd-4e3d-954a-757de8e42280 nodeName:}" failed. No retries permitted until 2026-06-10 12:44:59.609902353 +0000 UTC m=+489.820748318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/aa0cfac9-cacd-4e3d-954a-757de8e42280-tls-certs") pod "model-serving-api-5dd488fc48-cpfzp" (UID: "aa0cfac9-cacd-4e3d-954a-757de8e42280") : secret "model-serving-api-tls" not found Jun 10 12:44:59.128416 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.128380 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fh57\" (UniqueName: \"kubernetes.io/projected/aa0cfac9-cacd-4e3d-954a-757de8e42280-kube-api-access-8fh57\") pod \"model-serving-api-5dd488fc48-cpfzp\" (UID: \"aa0cfac9-cacd-4e3d-954a-757de8e42280\") " pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:44:59.617930 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.617866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa0cfac9-cacd-4e3d-954a-757de8e42280-tls-certs\") pod \"model-serving-api-5dd488fc48-cpfzp\" (UID: \"aa0cfac9-cacd-4e3d-954a-757de8e42280\") " pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:44:59.618177 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:44:59.618154 2582 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 10 12:44:59.618258 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:44:59.618239 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa0cfac9-cacd-4e3d-954a-757de8e42280-tls-certs podName:aa0cfac9-cacd-4e3d-954a-757de8e42280 nodeName:}" failed. No retries permitted until 2026-06-10 12:45:00.618217244 +0000 UTC m=+490.829063229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/aa0cfac9-cacd-4e3d-954a-757de8e42280-tls-certs") pod "model-serving-api-5dd488fc48-cpfzp" (UID: "aa0cfac9-cacd-4e3d-954a-757de8e42280") : secret "model-serving-api-tls" not found Jun 10 12:44:59.811942 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:44:59.811902 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:00.628946 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:00.628901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa0cfac9-cacd-4e3d-954a-757de8e42280-tls-certs\") pod \"model-serving-api-5dd488fc48-cpfzp\" (UID: \"aa0cfac9-cacd-4e3d-954a-757de8e42280\") " pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:45:00.633744 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:00.633690 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa0cfac9-cacd-4e3d-954a-757de8e42280-tls-certs\") pod \"model-serving-api-5dd488fc48-cpfzp\" (UID: \"aa0cfac9-cacd-4e3d-954a-757de8e42280\") " pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:45:00.681997 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:00.681957 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:45:01.089032 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:01.088995 2582 generic.go:356] "Generic (PLEG): container finished" podID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerID="668479e5a2dfb739dd11532f58dc48b8652d782d1cdfef812e08aec0e9672102" exitCode=0 Jun 10 12:45:01.089287 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:01.089088 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" event={"ID":"192ad9c9-58ae-4738-a56d-f16f8cf3dec7","Type":"ContainerDied","Data":"668479e5a2dfb739dd11532f58dc48b8652d782d1cdfef812e08aec0e9672102"} Jun 10 12:45:03.221702 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:03.221646 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-cpfzp"] Jun 10 12:45:04.116961 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:04.116929 2582 generic.go:356] "Generic (PLEG): container finished" podID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerID="c7b3c37daad1c03482c972cd9c949e31e3bec06ec7dde8c7aadcd2dd1820e765" exitCode=0 Jun 10 12:45:04.117142 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:04.117019 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" event={"ID":"192ad9c9-58ae-4738-a56d-f16f8cf3dec7","Type":"ContainerDied","Data":"c7b3c37daad1c03482c972cd9c949e31e3bec06ec7dde8c7aadcd2dd1820e765"} Jun 10 12:45:04.118718 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:04.118688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" event={"ID":"70700fe6-5cd4-4a4b-b646-ccc3c4eee484","Type":"ContainerStarted","Data":"af26edea6af454b02f44f21e66f50c56e754d1663124a190b6c75eae36c90e25"} Jun 10 12:45:04.120270 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:04.120194 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" event={"ID":"4a81b239-4f98-42d6-8de2-c1e069633bc5","Type":"ContainerStarted","Data":"241defdb303da362a2af94f79957303f51e0f223ddd0b90a1dd722e34ec3b5cf"} Jun 10 12:45:04.120368 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:04.120349 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:45:04.121435 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:04.121412 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" event={"ID":"aa0cfac9-cacd-4e3d-954a-757de8e42280","Type":"ContainerStarted","Data":"dd3c06d8bd775c14256d4ae23a57be603985ecc778dff88e548b91ffabeec4c3"} Jun 10 12:45:04.157987 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:04.157907 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" podStartSLOduration=2.845316972 podStartE2EDuration="7.157889622s" totalImagesPullingTime="4.31257265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:44:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:45:04.156985652 +0000 UTC m=+494.367831640" watchObservedRunningTime="2026-06-10 12:45:04.157889622 +0000 UTC m=+494.368735608" Jun 10 12:45:04.177154 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:04.177064 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7567468478-wz9pp" podStartSLOduration=2.368255988 podStartE2EDuration="7.177047437s" totalImagesPullingTime="4.808791449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:44:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:45:04.176303061 +0000 UTC m=+494.387149061" watchObservedRunningTime="2026-06-10 12:45:04.177047437 +0000 UTC m=+494.387893423" Jun 10 12:45:05.295882 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.295860 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:45:05.370940 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.370902 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwr65\" (UniqueName: \"kubernetes.io/projected/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-kube-api-access-zwr65\") pod \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " Jun 10 12:45:05.370940 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.370955 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-bundle\") pod \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " Jun 10 12:45:05.371227 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.371043 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-util\") pod \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\" (UID: \"192ad9c9-58ae-4738-a56d-f16f8cf3dec7\") " Jun 10 12:45:05.371861 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.371822 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-bundle" (OuterVolumeSpecName: "bundle") pod "192ad9c9-58ae-4738-a56d-f16f8cf3dec7" (UID: "192ad9c9-58ae-4738-a56d-f16f8cf3dec7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:45:05.373458 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.373329 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-kube-api-access-zwr65" (OuterVolumeSpecName: "kube-api-access-zwr65") pod "192ad9c9-58ae-4738-a56d-f16f8cf3dec7" (UID: "192ad9c9-58ae-4738-a56d-f16f8cf3dec7"). InnerVolumeSpecName "kube-api-access-zwr65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:45:05.375704 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.375676 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-util" (OuterVolumeSpecName: "util") pod "192ad9c9-58ae-4738-a56d-f16f8cf3dec7" (UID: "192ad9c9-58ae-4738-a56d-f16f8cf3dec7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:45:05.471955 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.471922 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:45:05.471955 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.471950 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwr65\" (UniqueName: \"kubernetes.io/projected/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-kube-api-access-zwr65\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:45:05.471955 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:05.471960 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192ad9c9-58ae-4738-a56d-f16f8cf3dec7-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:45:06.135020 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:06.134985 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" event={"ID":"aa0cfac9-cacd-4e3d-954a-757de8e42280","Type":"ContainerStarted","Data":"4f75826634d6bac927969052c36aa8e14b74a83e1f372f8b04101bf0b9ce7899"} Jun 10 12:45:06.135248 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:06.135124 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:45:06.137025 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:06.137004 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" Jun 10 12:45:06.137155 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:06.137008 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06pn4w6" event={"ID":"192ad9c9-58ae-4738-a56d-f16f8cf3dec7","Type":"ContainerDied","Data":"5f80165707be70c7fa8388b8ca04c7275980e8595724ad1aa05f5939c2118127"} Jun 10 12:45:06.137155 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:06.137124 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f80165707be70c7fa8388b8ca04c7275980e8595724ad1aa05f5939c2118127" Jun 10 12:45:06.188363 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:06.188306 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" podStartSLOduration=6.147298434 podStartE2EDuration="8.188287317s" totalImagesPullingTime="2.040988883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:45:06.187627615 +0000 UTC m=+496.398473602" watchObservedRunningTime="2026-06-10 12:45:06.188287317 +0000 UTC m=+496.399133303" Jun 10 12:45:09.213987 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.213953 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:09.320958 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.320915 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-vzc9b"] Jun 10 12:45:09.321433 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.321418 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerName="extract" Jun 10 12:45:09.321502 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.321436 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerName="extract" Jun 10 12:45:09.321502 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.321458 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerName="util" Jun 10 12:45:09.321502 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.321468 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerName="util" Jun 10 12:45:09.321502 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.321482 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerName="pull" Jun 10 12:45:09.321502 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.321491 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerName="pull" Jun 10 12:45:09.321648 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.321574 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="192ad9c9-58ae-4738-a56d-f16f8cf3dec7" containerName="extract" Jun 10 12:45:09.326867 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.326839 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d"] Jun 10 12:45:09.326989 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.326965 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:09.329973 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.329932 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mznql\"" Jun 10 12:45:09.329973 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.329969 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 10 12:45:09.331136 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.331115 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:09.333993 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.333971 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-c8sgj\"" Jun 10 12:45:09.334395 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.334354 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 10 12:45:09.342736 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.342576 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-vzc9b"] Jun 10 12:45:09.359566 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.359531 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d"] Jun 10 12:45:09.388544 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.388516 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:09.415996 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.415963 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec14a5bb-eca7-4851-9df2-61f5ab7bc066-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-w5q7d\" (UID: \"ec14a5bb-eca7-4851-9df2-61f5ab7bc066\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:09.416201 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.416003 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e-cert\") pod \"kserve-controller-manager-69fc866d76-vzc9b\" (UID: \"1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:09.416201 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.416070 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc9lb\" (UniqueName: \"kubernetes.io/projected/ec14a5bb-eca7-4851-9df2-61f5ab7bc066-kube-api-access-rc9lb\") pod \"llmisvc-controller-manager-864bfd7bc9-w5q7d\" (UID: \"ec14a5bb-eca7-4851-9df2-61f5ab7bc066\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:09.416308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.416250 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcx4m\" (UniqueName: \"kubernetes.io/projected/1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e-kube-api-access-wcx4m\") pod \"kserve-controller-manager-69fc866d76-vzc9b\" (UID: \"1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:09.517391 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.517282 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec14a5bb-eca7-4851-9df2-61f5ab7bc066-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-w5q7d\" (UID: \"ec14a5bb-eca7-4851-9df2-61f5ab7bc066\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:09.517391 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.517335 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e-cert\") pod \"kserve-controller-manager-69fc866d76-vzc9b\" (UID: \"1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:09.517391 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.517364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc9lb\" (UniqueName: \"kubernetes.io/projected/ec14a5bb-eca7-4851-9df2-61f5ab7bc066-kube-api-access-rc9lb\") pod \"llmisvc-controller-manager-864bfd7bc9-w5q7d\" (UID: \"ec14a5bb-eca7-4851-9df2-61f5ab7bc066\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:09.517672 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.517459 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcx4m\" (UniqueName: \"kubernetes.io/projected/1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e-kube-api-access-wcx4m\") pod \"kserve-controller-manager-69fc866d76-vzc9b\" (UID: \"1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:09.518191 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:45:09.517858 2582 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 10 12:45:09.518191 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:45:09.517950 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec14a5bb-eca7-4851-9df2-61f5ab7bc066-cert podName:ec14a5bb-eca7-4851-9df2-61f5ab7bc066 nodeName:}" failed. No retries permitted until 2026-06-10 12:45:10.01792782 +0000 UTC m=+500.228773803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec14a5bb-eca7-4851-9df2-61f5ab7bc066-cert") pod "llmisvc-controller-manager-864bfd7bc9-w5q7d" (UID: "ec14a5bb-eca7-4851-9df2-61f5ab7bc066") : secret "llmisvc-webhook-server-cert" not found Jun 10 12:45:09.521010 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.520866 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:09.521184 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.521163 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e-cert\") pod \"kserve-controller-manager-69fc866d76-vzc9b\" (UID: \"1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:09.529833 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.529772 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcx4m\" (UniqueName: \"kubernetes.io/projected/1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e-kube-api-access-wcx4m\") pod \"kserve-controller-manager-69fc866d76-vzc9b\" (UID: \"1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:09.531836 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.531802 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc9lb\" (UniqueName: \"kubernetes.io/projected/ec14a5bb-eca7-4851-9df2-61f5ab7bc066-kube-api-access-rc9lb\") pod \"llmisvc-controller-manager-864bfd7bc9-w5q7d\" (UID: \"ec14a5bb-eca7-4851-9df2-61f5ab7bc066\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:09.640323 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.640288 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:09.802051 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.802019 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-vzc9b"] Jun 10 12:45:09.803566 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:45:09.803534 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1673ed4a_a2cf_4f5c_9aad_5e17a1c0393e.slice/crio-f2345b24a3e0270188056f604e1114fa1fb371e33ccea31e7a4fc369e1c1fb66 WatchSource:0}: Error finding container f2345b24a3e0270188056f604e1114fa1fb371e33ccea31e7a4fc369e1c1fb66: Status 404 returned error can't find the container with id f2345b24a3e0270188056f604e1114fa1fb371e33ccea31e7a4fc369e1c1fb66 Jun 10 12:45:09.911486 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:09.911449 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:10.022705 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:10.022665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec14a5bb-eca7-4851-9df2-61f5ab7bc066-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-w5q7d\" (UID: \"ec14a5bb-eca7-4851-9df2-61f5ab7bc066\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:10.025285 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:10.025263 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec14a5bb-eca7-4851-9df2-61f5ab7bc066-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-w5q7d\" (UID: \"ec14a5bb-eca7-4851-9df2-61f5ab7bc066\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:10.158685 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:10.158649 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" event={"ID":"1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e","Type":"ContainerStarted","Data":"f2345b24a3e0270188056f604e1114fa1fb371e33ccea31e7a4fc369e1c1fb66"} Jun 10 12:45:10.246898 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:10.246865 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:10.387643 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:10.387615 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d"] Jun 10 12:45:10.389344 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:45:10.389309 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podec14a5bb_eca7_4851_9df2_61f5ab7bc066.slice/crio-47ca21957febcf8ff67c17029649227bf58267f2eaa3a51da50d598459572507 WatchSource:0}: Error finding container 47ca21957febcf8ff67c17029649227bf58267f2eaa3a51da50d598459572507: Status 404 returned error can't find the container with id 47ca21957febcf8ff67c17029649227bf58267f2eaa3a51da50d598459572507 Jun 10 12:45:10.602347 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:10.602317 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:10.796811 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:10.796778 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:10.959073 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:10.958993 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:11.167344 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:11.167303 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" event={"ID":"ec14a5bb-eca7-4851-9df2-61f5ab7bc066","Type":"ContainerStarted","Data":"47ca21957febcf8ff67c17029649227bf58267f2eaa3a51da50d598459572507"} Jun 10 12:45:12.590045 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.586503 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg"] Jun 10 12:45:12.594756 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.594730 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.606918 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.606888 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:45:12.607055 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.606942 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:45:12.608378 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.608358 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhnkq\"" Jun 10 12:45:12.642358 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.642325 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg"] Jun 10 12:45:12.752441 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.752401 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.752634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.752474 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.752634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.752538 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wj5w\" (UniqueName: \"kubernetes.io/projected/7dd4861e-35f6-43db-8636-5f632fe5ae44-kube-api-access-8wj5w\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.854078 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.853986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wj5w\" (UniqueName: \"kubernetes.io/projected/7dd4861e-35f6-43db-8636-5f632fe5ae44-kube-api-access-8wj5w\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.854299 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.854077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.854299 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.854157 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.854634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.854589 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.855349 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.855141 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.898205 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.898140 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wj5w\" (UniqueName: \"kubernetes.io/projected/7dd4861e-35f6-43db-8636-5f632fe5ae44-kube-api-access-8wj5w\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:12.971606 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:12.971014 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:13.183313 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:13.183225 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" event={"ID":"1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e","Type":"ContainerStarted","Data":"640c126e2e9369e59249029a0e92cd18cb97b6389abc494da28ba0a9b5c471a3"} Jun 10 12:45:13.183481 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:13.183408 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:13.319290 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:13.319239 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" podStartSLOduration=1.551798891 podStartE2EDuration="4.319219925s" totalImagesPullingTime="2.767421034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:45:13.318870375 +0000 UTC m=+503.529716363" watchObservedRunningTime="2026-06-10 12:45:13.319219925 +0000 UTC m=+503.530065911" Jun 10 12:45:13.463033 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:13.462958 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg"] Jun 10 12:45:13.473213 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:13.473190 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:13.595328 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:45:13.595294 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd4861e_35f6_43db_8636_5f632fe5ae44.slice/crio-6a5793c656fe41a7845b8f4e3f9cd6845d840ef4ea2e0475bf93ca9c0b106b24 WatchSource:0}: Error finding container 6a5793c656fe41a7845b8f4e3f9cd6845d840ef4ea2e0475bf93ca9c0b106b24: Status 404 returned error can't find the container with id 6a5793c656fe41a7845b8f4e3f9cd6845d840ef4ea2e0475bf93ca9c0b106b24 Jun 10 12:45:13.623358 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:13.623322 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:13.820721 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:13.820699 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:14.189981 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:14.189946 2582 generic.go:356] "Generic (PLEG): container finished" podID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerID="703fa0a7208ae89211c48e6eb301151ebec160a3e5cabce9dcad420b32429cb0" exitCode=0 Jun 10 12:45:14.190196 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:14.190051 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" event={"ID":"7dd4861e-35f6-43db-8636-5f632fe5ae44","Type":"ContainerDied","Data":"703fa0a7208ae89211c48e6eb301151ebec160a3e5cabce9dcad420b32429cb0"} Jun 10 12:45:14.190196 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:14.190158 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" event={"ID":"7dd4861e-35f6-43db-8636-5f632fe5ae44","Type":"ContainerStarted","Data":"6a5793c656fe41a7845b8f4e3f9cd6845d840ef4ea2e0475bf93ca9c0b106b24"} Jun 10 12:45:14.192031 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:14.192009 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" event={"ID":"ec14a5bb-eca7-4851-9df2-61f5ab7bc066","Type":"ContainerStarted","Data":"9e8a15d1e837730613efa987ae379f064a75f8d5b8aff0a78d516529037493e7"} Jun 10 12:45:14.251252 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:14.251198 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" podStartSLOduration=2.01133667 podStartE2EDuration="5.251180883s" totalImagesPullingTime="3.239844213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:45:14.250001219 +0000 UTC m=+504.460847207" watchObservedRunningTime="2026-06-10 12:45:14.251180883 +0000 UTC m=+504.462026872" Jun 10 12:45:14.530835 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:14.530762 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:15.131424 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:15.131398 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-zwbrs" Jun 10 12:45:15.197401 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:15.197374 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:16.108286 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.108256 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-fxc66"] Jun 10 12:45:16.134362 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.134333 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:16.138857 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.138834 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 10 12:45:16.139983 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.139963 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 10 12:45:16.140134 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.140003 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-hpr92\"" Jun 10 12:45:16.144749 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.144726 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-fxc66"] Jun 10 12:45:16.204913 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.204876 2582 generic.go:356] "Generic (PLEG): container finished" podID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerID="0c591ce3d68329fabaf7ef925f340509b417d47dc421096ee9b0c2dfebdea4d1" exitCode=0 Jun 10 12:45:16.205110 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.204960 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" event={"ID":"7dd4861e-35f6-43db-8636-5f632fe5ae44","Type":"ContainerDied","Data":"0c591ce3d68329fabaf7ef925f340509b417d47dc421096ee9b0c2dfebdea4d1"} Jun 10 12:45:16.282178 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.282093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/5f4af62a-89c8-4e44-82b5-204cc9a78a8a-operator-config\") pod \"servicemesh-operator3-57f65f65fb-fxc66\" (UID: \"5f4af62a-89c8-4e44-82b5-204cc9a78a8a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:16.282358 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.282217 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgcsd\" (UniqueName: \"kubernetes.io/projected/5f4af62a-89c8-4e44-82b5-204cc9a78a8a-kube-api-access-dgcsd\") pod \"servicemesh-operator3-57f65f65fb-fxc66\" (UID: \"5f4af62a-89c8-4e44-82b5-204cc9a78a8a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:16.383137 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.383029 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgcsd\" (UniqueName: \"kubernetes.io/projected/5f4af62a-89c8-4e44-82b5-204cc9a78a8a-kube-api-access-dgcsd\") pod \"servicemesh-operator3-57f65f65fb-fxc66\" (UID: \"5f4af62a-89c8-4e44-82b5-204cc9a78a8a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:16.383322 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.383165 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/5f4af62a-89c8-4e44-82b5-204cc9a78a8a-operator-config\") pod \"servicemesh-operator3-57f65f65fb-fxc66\" (UID: \"5f4af62a-89c8-4e44-82b5-204cc9a78a8a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:16.386465 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.386408 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/5f4af62a-89c8-4e44-82b5-204cc9a78a8a-operator-config\") pod \"servicemesh-operator3-57f65f65fb-fxc66\" (UID: \"5f4af62a-89c8-4e44-82b5-204cc9a78a8a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:16.396849 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.396819 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgcsd\" (UniqueName: \"kubernetes.io/projected/5f4af62a-89c8-4e44-82b5-204cc9a78a8a-kube-api-access-dgcsd\") pod \"servicemesh-operator3-57f65f65fb-fxc66\" (UID: \"5f4af62a-89c8-4e44-82b5-204cc9a78a8a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:16.444694 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.444639 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:16.632077 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:16.632042 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-fxc66"] Jun 10 12:45:16.633787 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:45:16.633720 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4af62a_89c8_4e44_82b5_204cc9a78a8a.slice/crio-b050cb78b141b7405e964f2a7ea3cab9069cd5498302c83f5c5c985f5ea22f0f WatchSource:0}: Error finding container b050cb78b141b7405e964f2a7ea3cab9069cd5498302c83f5c5c985f5ea22f0f: Status 404 returned error can't find the container with id b050cb78b141b7405e964f2a7ea3cab9069cd5498302c83f5c5c985f5ea22f0f Jun 10 12:45:17.146768 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:17.146734 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-cpfzp" Jun 10 12:45:17.212779 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:17.212749 2582 generic.go:356] "Generic (PLEG): container finished" podID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerID="494aa2a0c085b5ae6a07d457b7bc19875f09f23a28f5bda1eae9d11f7aed2334" exitCode=0 Jun 10 12:45:17.212942 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:17.212828 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" event={"ID":"7dd4861e-35f6-43db-8636-5f632fe5ae44","Type":"ContainerDied","Data":"494aa2a0c085b5ae6a07d457b7bc19875f09f23a28f5bda1eae9d11f7aed2334"} Jun 10 12:45:17.214458 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:17.214439 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" event={"ID":"5f4af62a-89c8-4e44-82b5-204cc9a78a8a","Type":"ContainerStarted","Data":"b050cb78b141b7405e964f2a7ea3cab9069cd5498302c83f5c5c985f5ea22f0f"} Jun 10 12:45:18.388694 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.388666 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:18.505271 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.505182 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-util\") pod \"7dd4861e-35f6-43db-8636-5f632fe5ae44\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " Jun 10 12:45:18.505271 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.505247 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-bundle\") pod \"7dd4861e-35f6-43db-8636-5f632fe5ae44\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " Jun 10 12:45:18.505492 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.505277 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wj5w\" (UniqueName: \"kubernetes.io/projected/7dd4861e-35f6-43db-8636-5f632fe5ae44-kube-api-access-8wj5w\") pod \"7dd4861e-35f6-43db-8636-5f632fe5ae44\" (UID: \"7dd4861e-35f6-43db-8636-5f632fe5ae44\") " Jun 10 12:45:18.506414 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.506383 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-bundle" (OuterVolumeSpecName: "bundle") pod "7dd4861e-35f6-43db-8636-5f632fe5ae44" (UID: "7dd4861e-35f6-43db-8636-5f632fe5ae44"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:45:18.508250 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.508222 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd4861e-35f6-43db-8636-5f632fe5ae44-kube-api-access-8wj5w" (OuterVolumeSpecName: "kube-api-access-8wj5w") pod "7dd4861e-35f6-43db-8636-5f632fe5ae44" (UID: "7dd4861e-35f6-43db-8636-5f632fe5ae44"). InnerVolumeSpecName "kube-api-access-8wj5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:45:18.511754 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.511717 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-util" (OuterVolumeSpecName: "util") pod "7dd4861e-35f6-43db-8636-5f632fe5ae44" (UID: "7dd4861e-35f6-43db-8636-5f632fe5ae44"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:45:18.606566 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.606520 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:45:18.606566 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.606559 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7dd4861e-35f6-43db-8636-5f632fe5ae44-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:45:18.606773 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:18.606576 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wj5w\" (UniqueName: \"kubernetes.io/projected/7dd4861e-35f6-43db-8636-5f632fe5ae44-kube-api-access-8wj5w\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:45:19.231665 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:19.231633 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" Jun 10 12:45:19.232005 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:19.231632 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762awhccg" event={"ID":"7dd4861e-35f6-43db-8636-5f632fe5ae44","Type":"ContainerDied","Data":"6a5793c656fe41a7845b8f4e3f9cd6845d840ef4ea2e0475bf93ca9c0b106b24"} Jun 10 12:45:19.232005 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:19.231778 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a5793c656fe41a7845b8f4e3f9cd6845d840ef4ea2e0475bf93ca9c0b106b24" Jun 10 12:45:20.238833 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:20.238796 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" event={"ID":"5f4af62a-89c8-4e44-82b5-204cc9a78a8a","Type":"ContainerStarted","Data":"882959ef33e460e2a08b6b9cb420c470d3d92e2f8cafaa1c2ff4ddf08aa8fa64"} Jun 10 12:45:20.239241 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:20.238929 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:28.090555 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.090498 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" podStartSLOduration=9.187808184 podStartE2EDuration="12.090480189s" totalImagesPullingTime="2.902672005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:45:20.306345074 +0000 UTC m=+510.517191060" watchObservedRunningTime="2026-06-10 12:45:28.090480189 +0000 UTC m=+518.301326183" Jun 10 12:45:28.091691 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.091674 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-f676969f4-7fsr8"] Jun 10 12:45:28.092121 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.092089 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerName="extract" Jun 10 12:45:28.092174 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.092125 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerName="extract" Jun 10 12:45:28.092174 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.092149 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerName="pull" Jun 10 12:45:28.092174 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.092155 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerName="pull" Jun 10 12:45:28.092174 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.092166 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerName="util" Jun 10 12:45:28.092174 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.092171 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerName="util" Jun 10 12:45:28.092329 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.092239 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dd4861e-35f6-43db-8636-5f632fe5ae44" containerName="extract" Jun 10 12:45:28.095569 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.095549 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" Jun 10 12:45:28.098448 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.098426 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 10 12:45:28.098582 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.098426 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7cc8g\"" Jun 10 12:45:28.109394 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.109367 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-f676969f4-7fsr8"] Jun 10 12:45:28.139411 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.139373 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:28.188973 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.188939 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/4ad6ffbe-3a4c-455c-8a24-472c123a04bb-redirect-config\") pod \"dashboard-redirect-f676969f4-7fsr8\" (UID: \"4ad6ffbe-3a4c-455c-8a24-472c123a04bb\") " pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" Jun 10 12:45:28.188973 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.188978 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrjcs\" (UniqueName: \"kubernetes.io/projected/4ad6ffbe-3a4c-455c-8a24-472c123a04bb-kube-api-access-zrjcs\") pod \"dashboard-redirect-f676969f4-7fsr8\" (UID: \"4ad6ffbe-3a4c-455c-8a24-472c123a04bb\") " pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" Jun 10 12:45:28.289989 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.289944 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrjcs\" (UniqueName: \"kubernetes.io/projected/4ad6ffbe-3a4c-455c-8a24-472c123a04bb-kube-api-access-zrjcs\") pod \"dashboard-redirect-f676969f4-7fsr8\" (UID: \"4ad6ffbe-3a4c-455c-8a24-472c123a04bb\") " pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" Jun 10 12:45:28.290234 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.290122 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/4ad6ffbe-3a4c-455c-8a24-472c123a04bb-redirect-config\") pod \"dashboard-redirect-f676969f4-7fsr8\" (UID: \"4ad6ffbe-3a4c-455c-8a24-472c123a04bb\") " pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" Jun 10 12:45:28.290792 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.290768 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/4ad6ffbe-3a4c-455c-8a24-472c123a04bb-redirect-config\") pod \"dashboard-redirect-f676969f4-7fsr8\" (UID: \"4ad6ffbe-3a4c-455c-8a24-472c123a04bb\") " pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" Jun 10 12:45:28.299803 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.299778 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrjcs\" (UniqueName: \"kubernetes.io/projected/4ad6ffbe-3a4c-455c-8a24-472c123a04bb-kube-api-access-zrjcs\") pod \"dashboard-redirect-f676969f4-7fsr8\" (UID: \"4ad6ffbe-3a4c-455c-8a24-472c123a04bb\") " pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" Jun 10 12:45:28.407588 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.407505 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" Jun 10 12:45:28.548009 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:28.547986 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-f676969f4-7fsr8"] Jun 10 12:45:28.549740 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:45:28.549708 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad6ffbe_3a4c_455c_8a24_472c123a04bb.slice/crio-83ae1cdc210cfa40cef950ca236f1ad577b6ca2071b2c31be77403dcbc1845c6 WatchSource:0}: Error finding container 83ae1cdc210cfa40cef950ca236f1ad577b6ca2071b2c31be77403dcbc1845c6: Status 404 returned error can't find the container with id 83ae1cdc210cfa40cef950ca236f1ad577b6ca2071b2c31be77403dcbc1845c6 Jun 10 12:45:29.290745 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:29.290713 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" event={"ID":"4ad6ffbe-3a4c-455c-8a24-472c123a04bb","Type":"ContainerStarted","Data":"83ae1cdc210cfa40cef950ca236f1ad577b6ca2071b2c31be77403dcbc1845c6"} Jun 10 12:45:31.246117 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:31.246063 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-fxc66" Jun 10 12:45:33.316960 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:33.316920 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" event={"ID":"4ad6ffbe-3a4c-455c-8a24-472c123a04bb","Type":"ContainerStarted","Data":"ea0f77ecceed071f182e9de021c57302edfb3933743b939c4f9b1186632062d7"} Jun 10 12:45:33.353252 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:33.353197 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-f676969f4-7fsr8" podStartSLOduration=0.717810629 podStartE2EDuration="5.353180339s" totalImagesPullingTime="4.63536971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:45:33.351923412 +0000 UTC m=+523.562769398" watchObservedRunningTime="2026-06-10 12:45:33.353180339 +0000 UTC m=+523.564026326" Jun 10 12:45:40.862715 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.862677 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9"] Jun 10 12:45:40.867680 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.867654 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:40.867931 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.867916 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:40.870735 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.870714 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 10 12:45:40.870735 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.870731 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 10 12:45:40.870924 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.870771 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 10 12:45:40.871048 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.871035 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kzc5d\"" Jun 10 12:45:40.871869 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.871853 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 10 12:45:40.878589 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.878566 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9"] Jun 10 12:45:40.916281 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.916241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:40.916468 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.916306 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:40.916468 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.916330 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:40.916468 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.916376 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:40.916468 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.916431 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnbcm\" (UniqueName: \"kubernetes.io/projected/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-kube-api-access-cnbcm\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:40.916638 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.916476 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:40.916638 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.916538 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:40.953339 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:40.953309 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:41.017776 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.017739 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.017953 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.017795 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.017953 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.017821 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.017953 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.017839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.017953 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.017858 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnbcm\" (UniqueName: \"kubernetes.io/projected/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-kube-api-access-cnbcm\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.017953 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.017884 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.018261 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.017967 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.018669 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.018620 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.020653 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.020623 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.020781 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.020758 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.020849 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.020762 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.020909 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.020846 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.027455 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.027425 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.027585 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.027480 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnbcm\" (UniqueName: \"kubernetes.io/projected/e2def4ea-79e3-4a9f-884c-a3fd1d95554a-kube-api-access-cnbcm\") pod \"istiod-openshift-gateway-75c67f8887-pbvl9\" (UID: \"e2def4ea-79e3-4a9f-884c-a3fd1d95554a\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.178264 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.178153 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:41.328651 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.328625 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9"] Jun 10 12:45:41.331542 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:45:41.331407 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2def4ea_79e3_4a9f_884c_a3fd1d95554a.slice/crio-39228970c52b2491748b50e2ac9783c2a6a8be48c20809f0031dd06512b77bde WatchSource:0}: Error finding container 39228970c52b2491748b50e2ac9783c2a6a8be48c20809f0031dd06512b77bde: Status 404 returned error can't find the container with id 39228970c52b2491748b50e2ac9783c2a6a8be48c20809f0031dd06512b77bde Jun 10 12:45:41.361592 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.361558 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" event={"ID":"e2def4ea-79e3-4a9f-884c-a3fd1d95554a","Type":"ContainerStarted","Data":"39228970c52b2491748b50e2ac9783c2a6a8be48c20809f0031dd06512b77bde"} Jun 10 12:45:41.434308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:41.434221 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:44.196902 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:44.196871 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-vzc9b" Jun 10 12:45:46.207220 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:46.207188 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-w5q7d" Jun 10 12:45:54.374456 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:54.374420 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 10 12:45:54.374712 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:54.374486 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 10 12:45:55.246347 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.246308 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:55.345042 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.344988 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:55.438525 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.438491 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" event={"ID":"e2def4ea-79e3-4a9f-884c-a3fd1d95554a","Type":"ContainerStarted","Data":"c674353724716ca8df99ed2ea4d4f054c946bd7afb7ec0143ab8e9bd301e34ea"} Jun 10 12:45:55.438989 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.438655 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:55.440545 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.440520 2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-pbvl9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 10 12:45:55.440655 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.440569 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" podUID="e2def4ea-79e3-4a9f-884c-a3fd1d95554a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 10 12:45:55.464232 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.464174 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" podStartSLOduration=2.423024367 podStartE2EDuration="15.464158427s" totalImagesPullingTime="13.04113406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:45:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:45:55.462979669 +0000 UTC m=+545.673825655" watchObservedRunningTime="2026-06-10 12:45:55.464158427 +0000 UTC m=+545.675004414" Jun 10 12:45:55.540015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.539927 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:55.668029 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.667988 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:55.780164 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:55.780120 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:56.445353 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:56.445322 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-pbvl9" Jun 10 12:45:57.012209 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:57.012177 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:58.239359 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:58.239328 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:58.338465 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:58.338418 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:45:58.421025 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:45:58.420987 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:10.312110 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:10.312075 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:13.560134 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.557470 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l"] Jun 10 12:46:13.581970 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.581938 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l"] Jun 10 12:46:13.582176 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.582069 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.585627 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.585604 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5qnsx\"" Jun 10 12:46:13.585753 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.585604 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 10 12:46:13.586822 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.586802 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 10 12:46:13.619570 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.619541 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.619728 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.619646 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.619728 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.619692 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c4wq\" (UniqueName: \"kubernetes.io/projected/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-kube-api-access-5c4wq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.720471 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.720436 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.720680 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.720486 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4wq\" (UniqueName: \"kubernetes.io/projected/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-kube-api-access-5c4wq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.720680 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.720517 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.720921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.720895 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.720990 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.720934 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.734414 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.734387 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c4wq\" (UniqueName: \"kubernetes.io/projected/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-kube-api-access-5c4wq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:13.892421 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:13.892332 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:14.054008 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.053979 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l"] Jun 10 12:46:14.055681 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:46:14.055654 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d63aba_abc2_4dc1_933c_71f1d0cb29f7.slice/crio-a422d1d5b658767ed2911c57b363e3c1187789f52764c704bd3b3b33d5975ce0 WatchSource:0}: Error finding container a422d1d5b658767ed2911c57b363e3c1187789f52764c704bd3b3b33d5975ce0: Status 404 returned error can't find the container with id a422d1d5b658767ed2911c57b363e3c1187789f52764c704bd3b3b33d5975ce0 Jun 10 12:46:14.182250 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.182221 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr"] Jun 10 12:46:14.185949 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.185928 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.202495 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.202471 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr"] Jun 10 12:46:14.224903 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.224871 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctsz4\" (UniqueName: \"kubernetes.io/projected/527361d7-e4c6-4f05-90b1-560e11228c77-kube-api-access-ctsz4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.225081 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.224922 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.225081 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.225012 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.325494 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.325455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctsz4\" (UniqueName: \"kubernetes.io/projected/527361d7-e4c6-4f05-90b1-560e11228c77-kube-api-access-ctsz4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.325705 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.325511 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.325705 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.325554 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.325872 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.325855 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.325922 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.325895 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.336236 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.336202 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctsz4\" (UniqueName: \"kubernetes.io/projected/527361d7-e4c6-4f05-90b1-560e11228c77-kube-api-access-ctsz4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.543704 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.543605 2582 generic.go:356] "Generic (PLEG): container finished" podID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerID="af56fb8912ce90c5160f62d8ac06d2ac71f4c3710c15a4aec531f74d738c3a68" exitCode=0 Jun 10 12:46:14.543704 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.543688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" event={"ID":"03d63aba-abc2-4dc1-933c-71f1d0cb29f7","Type":"ContainerDied","Data":"af56fb8912ce90c5160f62d8ac06d2ac71f4c3710c15a4aec531f74d738c3a68"} Jun 10 12:46:14.543903 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.543739 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" event={"ID":"03d63aba-abc2-4dc1-933c-71f1d0cb29f7","Type":"ContainerStarted","Data":"a422d1d5b658767ed2911c57b363e3c1187789f52764c704bd3b3b33d5975ce0"} Jun 10 12:46:14.544450 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.544432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:14.691118 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.691070 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr"] Jun 10 12:46:14.692252 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:46:14.692219 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527361d7_e4c6_4f05_90b1_560e11228c77.slice/crio-a2432dc7706e6d822eef17d5d17320af9920f3d0fe25555972a40806c02b2924 WatchSource:0}: Error finding container a2432dc7706e6d822eef17d5d17320af9920f3d0fe25555972a40806c02b2924: Status 404 returned error can't find the container with id a2432dc7706e6d822eef17d5d17320af9920f3d0fe25555972a40806c02b2924 Jun 10 12:46:14.765827 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.765799 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl"] Jun 10 12:46:14.769563 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.769544 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:14.780677 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.780645 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl"] Jun 10 12:46:14.830321 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.830286 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:14.830488 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.830358 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx9vq\" (UniqueName: \"kubernetes.io/projected/a1c12211-3534-4788-b5bd-39a0be5843fd-kube-api-access-qx9vq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:14.830488 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.830382 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:14.931689 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.931644 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:14.931840 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.931732 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx9vq\" (UniqueName: \"kubernetes.io/projected/a1c12211-3534-4788-b5bd-39a0be5843fd-kube-api-access-qx9vq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:14.931840 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.931765 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:14.932153 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.932134 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:14.932196 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.932165 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:14.945661 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:14.945633 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx9vq\" (UniqueName: \"kubernetes.io/projected/a1c12211-3534-4788-b5bd-39a0be5843fd-kube-api-access-qx9vq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:15.080010 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.079980 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:15.185422 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.184824 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5"] Jun 10 12:46:15.190216 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.190192 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.203939 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.203907 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5"] Jun 10 12:46:15.233955 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.233927 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vvw5\" (UniqueName: \"kubernetes.io/projected/23d17881-2f4f-4655-83df-8979581837e6-kube-api-access-4vvw5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.234091 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.233965 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.234091 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.234020 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.277082 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.277059 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl"] Jun 10 12:46:15.319971 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:46:15.319940 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c12211_3534_4788_b5bd_39a0be5843fd.slice/crio-d801ab749253f080b18fe4bc17c38c29757bb945c7c940f6598f2dfa2194f281 WatchSource:0}: Error finding container d801ab749253f080b18fe4bc17c38c29757bb945c7c940f6598f2dfa2194f281: Status 404 returned error can't find the container with id d801ab749253f080b18fe4bc17c38c29757bb945c7c940f6598f2dfa2194f281 Jun 10 12:46:15.334763 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.334741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vvw5\" (UniqueName: \"kubernetes.io/projected/23d17881-2f4f-4655-83df-8979581837e6-kube-api-access-4vvw5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.334828 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.334781 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.334871 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.334855 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.335263 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.335246 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.335330 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.335308 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.350935 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.350911 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vvw5\" (UniqueName: \"kubernetes.io/projected/23d17881-2f4f-4655-83df-8979581837e6-kube-api-access-4vvw5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.522668 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.522630 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:15.551633 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.551600 2582 generic.go:356] "Generic (PLEG): container finished" podID="527361d7-e4c6-4f05-90b1-560e11228c77" containerID="853fd9d1ae1913a8b3b2c76c01af4a4faf05a0796253c2295b7b59688a6d7972" exitCode=0 Jun 10 12:46:15.551824 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.551686 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" event={"ID":"527361d7-e4c6-4f05-90b1-560e11228c77","Type":"ContainerDied","Data":"853fd9d1ae1913a8b3b2c76c01af4a4faf05a0796253c2295b7b59688a6d7972"} Jun 10 12:46:15.551824 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.551739 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" event={"ID":"527361d7-e4c6-4f05-90b1-560e11228c77","Type":"ContainerStarted","Data":"a2432dc7706e6d822eef17d5d17320af9920f3d0fe25555972a40806c02b2924"} Jun 10 12:46:15.553629 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.553605 2582 generic.go:356] "Generic (PLEG): container finished" podID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerID="e8b3d0911f8c9ad59a89ea8914ded0da466e16f6e8fb0a4f732f3df3584e5728" exitCode=0 Jun 10 12:46:15.553755 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.553686 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" event={"ID":"a1c12211-3534-4788-b5bd-39a0be5843fd","Type":"ContainerDied","Data":"e8b3d0911f8c9ad59a89ea8914ded0da466e16f6e8fb0a4f732f3df3584e5728"} Jun 10 12:46:15.553755 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.553749 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" event={"ID":"a1c12211-3534-4788-b5bd-39a0be5843fd","Type":"ContainerStarted","Data":"d801ab749253f080b18fe4bc17c38c29757bb945c7c940f6598f2dfa2194f281"} Jun 10 12:46:15.555718 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.555692 2582 generic.go:356] "Generic (PLEG): container finished" podID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerID="6cbeacde691fc29e1f7745b948755ac1c35a139207afd6ce933eddbc5c9bac68" exitCode=0 Jun 10 12:46:15.555824 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.555732 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" event={"ID":"03d63aba-abc2-4dc1-933c-71f1d0cb29f7","Type":"ContainerDied","Data":"6cbeacde691fc29e1f7745b948755ac1c35a139207afd6ce933eddbc5c9bac68"} Jun 10 12:46:15.679354 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:15.679317 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5"] Jun 10 12:46:15.680161 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:46:15.680090 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d17881_2f4f_4655_83df_8979581837e6.slice/crio-32342560b175a4a97a375ea8bebf482597c4bab02c11202611d9e833177fc875 WatchSource:0}: Error finding container 32342560b175a4a97a375ea8bebf482597c4bab02c11202611d9e833177fc875: Status 404 returned error can't find the container with id 32342560b175a4a97a375ea8bebf482597c4bab02c11202611d9e833177fc875 Jun 10 12:46:16.563168 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:16.563129 2582 generic.go:356] "Generic (PLEG): container finished" podID="527361d7-e4c6-4f05-90b1-560e11228c77" containerID="ee65ff04f7b9d0f8d28c332befded7c2377b084c737310a0cfdb47dbb9fbd82b" exitCode=0 Jun 10 12:46:16.563541 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:16.563213 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" event={"ID":"527361d7-e4c6-4f05-90b1-560e11228c77","Type":"ContainerDied","Data":"ee65ff04f7b9d0f8d28c332befded7c2377b084c737310a0cfdb47dbb9fbd82b"} Jun 10 12:46:16.564811 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:16.564790 2582 generic.go:356] "Generic (PLEG): container finished" podID="23d17881-2f4f-4655-83df-8979581837e6" containerID="401c19b4f11e97e939db1456e8c1b412da204ddd464ffd1723bb0da63b485675" exitCode=0 Jun 10 12:46:16.564884 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:16.564869 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" event={"ID":"23d17881-2f4f-4655-83df-8979581837e6","Type":"ContainerDied","Data":"401c19b4f11e97e939db1456e8c1b412da204ddd464ffd1723bb0da63b485675"} Jun 10 12:46:16.564948 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:16.564911 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" event={"ID":"23d17881-2f4f-4655-83df-8979581837e6","Type":"ContainerStarted","Data":"32342560b175a4a97a375ea8bebf482597c4bab02c11202611d9e833177fc875"} Jun 10 12:46:16.566888 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:16.566865 2582 generic.go:356] "Generic (PLEG): container finished" podID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerID="18bb59703ecd1dc4c53ae03d587c503cdbaad690a71a9dc39e51e90bffbc95b7" exitCode=0 Jun 10 12:46:16.567005 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:16.566912 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" event={"ID":"a1c12211-3534-4788-b5bd-39a0be5843fd","Type":"ContainerDied","Data":"18bb59703ecd1dc4c53ae03d587c503cdbaad690a71a9dc39e51e90bffbc95b7"} Jun 10 12:46:16.569472 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:16.569372 2582 generic.go:356] "Generic (PLEG): container finished" podID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerID="4ba0edef075d905df20d76be3e5de04c20e8740c55486cea3178642c4a76f5d4" exitCode=0 Jun 10 12:46:16.569472 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:16.569458 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" event={"ID":"03d63aba-abc2-4dc1-933c-71f1d0cb29f7","Type":"ContainerDied","Data":"4ba0edef075d905df20d76be3e5de04c20e8740c55486cea3178642c4a76f5d4"} Jun 10 12:46:17.576746 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.576714 2582 generic.go:356] "Generic (PLEG): container finished" podID="23d17881-2f4f-4655-83df-8979581837e6" containerID="6786fcd3f5caa44054fe4e9c428eb16844868660e39c80c090958314db694c59" exitCode=0 Jun 10 12:46:17.577165 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.576793 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" event={"ID":"23d17881-2f4f-4655-83df-8979581837e6","Type":"ContainerDied","Data":"6786fcd3f5caa44054fe4e9c428eb16844868660e39c80c090958314db694c59"} Jun 10 12:46:17.579332 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.579301 2582 generic.go:356] "Generic (PLEG): container finished" podID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerID="8842f7df2be333654f011891b053ba5ebba4d9b1903155bcfb2c39bcdeee2574" exitCode=0 Jun 10 12:46:17.579420 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.579376 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" event={"ID":"a1c12211-3534-4788-b5bd-39a0be5843fd","Type":"ContainerDied","Data":"8842f7df2be333654f011891b053ba5ebba4d9b1903155bcfb2c39bcdeee2574"} Jun 10 12:46:17.581457 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.581437 2582 generic.go:356] "Generic (PLEG): container finished" podID="527361d7-e4c6-4f05-90b1-560e11228c77" containerID="45398f96a7d787727a0c4bb041a3a62db572e8d846fb42eae1bcb118448d9b63" exitCode=0 Jun 10 12:46:17.581540 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.581502 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" event={"ID":"527361d7-e4c6-4f05-90b1-560e11228c77","Type":"ContainerDied","Data":"45398f96a7d787727a0c4bb041a3a62db572e8d846fb42eae1bcb118448d9b63"} Jun 10 12:46:17.721774 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.721747 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:17.765339 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.765312 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-bundle\") pod \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " Jun 10 12:46:17.765490 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.765352 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-util\") pod \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " Jun 10 12:46:17.765490 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.765460 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c4wq\" (UniqueName: \"kubernetes.io/projected/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-kube-api-access-5c4wq\") pod \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\" (UID: \"03d63aba-abc2-4dc1-933c-71f1d0cb29f7\") " Jun 10 12:46:17.765912 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.765881 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-bundle" (OuterVolumeSpecName: "bundle") pod "03d63aba-abc2-4dc1-933c-71f1d0cb29f7" (UID: "03d63aba-abc2-4dc1-933c-71f1d0cb29f7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:46:17.767741 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.767710 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-kube-api-access-5c4wq" (OuterVolumeSpecName: "kube-api-access-5c4wq") pod "03d63aba-abc2-4dc1-933c-71f1d0cb29f7" (UID: "03d63aba-abc2-4dc1-933c-71f1d0cb29f7"). InnerVolumeSpecName "kube-api-access-5c4wq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:46:17.770654 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.770627 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-util" (OuterVolumeSpecName: "util") pod "03d63aba-abc2-4dc1-933c-71f1d0cb29f7" (UID: "03d63aba-abc2-4dc1-933c-71f1d0cb29f7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:46:17.867058 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.866973 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c4wq\" (UniqueName: \"kubernetes.io/projected/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-kube-api-access-5c4wq\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:17.867058 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.867003 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:17.867058 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:17.867013 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/03d63aba-abc2-4dc1-933c-71f1d0cb29f7-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:18.588649 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.588610 2582 generic.go:356] "Generic (PLEG): container finished" podID="23d17881-2f4f-4655-83df-8979581837e6" containerID="4cef4fa22e87411022cdec5bbae1da652e1f729572ba8951889a6508bcccc097" exitCode=0 Jun 10 12:46:18.589020 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.588700 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" event={"ID":"23d17881-2f4f-4655-83df-8979581837e6","Type":"ContainerDied","Data":"4cef4fa22e87411022cdec5bbae1da652e1f729572ba8951889a6508bcccc097"} Jun 10 12:46:18.590619 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.590601 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" Jun 10 12:46:18.590736 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.590648 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l" event={"ID":"03d63aba-abc2-4dc1-933c-71f1d0cb29f7","Type":"ContainerDied","Data":"a422d1d5b658767ed2911c57b363e3c1187789f52764c704bd3b3b33d5975ce0"} Jun 10 12:46:18.590736 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.590686 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a422d1d5b658767ed2911c57b363e3c1187789f52764c704bd3b3b33d5975ce0" Jun 10 12:46:18.742525 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.742498 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:18.776072 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.776045 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:18.877301 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.877216 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-bundle\") pod \"a1c12211-3534-4788-b5bd-39a0be5843fd\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " Jun 10 12:46:18.877301 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.877255 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-bundle\") pod \"527361d7-e4c6-4f05-90b1-560e11228c77\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " Jun 10 12:46:18.877518 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.877348 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-util\") pod \"a1c12211-3534-4788-b5bd-39a0be5843fd\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " Jun 10 12:46:18.877518 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.877396 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-util\") pod \"527361d7-e4c6-4f05-90b1-560e11228c77\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " Jun 10 12:46:18.877518 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.877430 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctsz4\" (UniqueName: \"kubernetes.io/projected/527361d7-e4c6-4f05-90b1-560e11228c77-kube-api-access-ctsz4\") pod \"527361d7-e4c6-4f05-90b1-560e11228c77\" (UID: \"527361d7-e4c6-4f05-90b1-560e11228c77\") " Jun 10 12:46:18.877518 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.877482 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx9vq\" (UniqueName: \"kubernetes.io/projected/a1c12211-3534-4788-b5bd-39a0be5843fd-kube-api-access-qx9vq\") pod \"a1c12211-3534-4788-b5bd-39a0be5843fd\" (UID: \"a1c12211-3534-4788-b5bd-39a0be5843fd\") " Jun 10 12:46:18.880413 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.878076 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-bundle" (OuterVolumeSpecName: "bundle") pod "a1c12211-3534-4788-b5bd-39a0be5843fd" (UID: "a1c12211-3534-4788-b5bd-39a0be5843fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:46:18.880413 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.878333 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:18.880413 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.878344 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-bundle" (OuterVolumeSpecName: "bundle") pod "527361d7-e4c6-4f05-90b1-560e11228c77" (UID: "527361d7-e4c6-4f05-90b1-560e11228c77"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:46:18.889995 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.885246 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c12211-3534-4788-b5bd-39a0be5843fd-kube-api-access-qx9vq" (OuterVolumeSpecName: "kube-api-access-qx9vq") pod "a1c12211-3534-4788-b5bd-39a0be5843fd" (UID: "a1c12211-3534-4788-b5bd-39a0be5843fd"). InnerVolumeSpecName "kube-api-access-qx9vq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:46:18.889995 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.885660 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-util" (OuterVolumeSpecName: "util") pod "527361d7-e4c6-4f05-90b1-560e11228c77" (UID: "527361d7-e4c6-4f05-90b1-560e11228c77"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:46:18.889995 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.886964 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/527361d7-e4c6-4f05-90b1-560e11228c77-kube-api-access-ctsz4" (OuterVolumeSpecName: "kube-api-access-ctsz4") pod "527361d7-e4c6-4f05-90b1-560e11228c77" (UID: "527361d7-e4c6-4f05-90b1-560e11228c77"). InnerVolumeSpecName "kube-api-access-ctsz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:46:18.889995 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.886991 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-util" (OuterVolumeSpecName: "util") pod "a1c12211-3534-4788-b5bd-39a0be5843fd" (UID: "a1c12211-3534-4788-b5bd-39a0be5843fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:46:18.979149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.979084 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1c12211-3534-4788-b5bd-39a0be5843fd-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:18.979149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.979141 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:18.979347 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.979171 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctsz4\" (UniqueName: \"kubernetes.io/projected/527361d7-e4c6-4f05-90b1-560e11228c77-kube-api-access-ctsz4\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:18.979347 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.979182 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx9vq\" (UniqueName: \"kubernetes.io/projected/a1c12211-3534-4788-b5bd-39a0be5843fd-kube-api-access-qx9vq\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:18.979347 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:18.979192 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/527361d7-e4c6-4f05-90b1-560e11228c77-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:19.599015 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.598980 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" Jun 10 12:46:19.599406 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.599011 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl" event={"ID":"a1c12211-3534-4788-b5bd-39a0be5843fd","Type":"ContainerDied","Data":"d801ab749253f080b18fe4bc17c38c29757bb945c7c940f6598f2dfa2194f281"} Jun 10 12:46:19.599406 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.599068 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d801ab749253f080b18fe4bc17c38c29757bb945c7c940f6598f2dfa2194f281" Jun 10 12:46:19.600964 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.600945 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" Jun 10 12:46:19.601087 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.600966 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr" event={"ID":"527361d7-e4c6-4f05-90b1-560e11228c77","Type":"ContainerDied","Data":"a2432dc7706e6d822eef17d5d17320af9920f3d0fe25555972a40806c02b2924"} Jun 10 12:46:19.601087 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.601011 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2432dc7706e6d822eef17d5d17320af9920f3d0fe25555972a40806c02b2924" Jun 10 12:46:19.740646 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.740621 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:19.787258 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.787231 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vvw5\" (UniqueName: \"kubernetes.io/projected/23d17881-2f4f-4655-83df-8979581837e6-kube-api-access-4vvw5\") pod \"23d17881-2f4f-4655-83df-8979581837e6\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " Jun 10 12:46:19.787425 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.787363 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-bundle\") pod \"23d17881-2f4f-4655-83df-8979581837e6\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " Jun 10 12:46:19.787425 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.787421 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-util\") pod \"23d17881-2f4f-4655-83df-8979581837e6\" (UID: \"23d17881-2f4f-4655-83df-8979581837e6\") " Jun 10 12:46:19.788009 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.787981 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-bundle" (OuterVolumeSpecName: "bundle") pod "23d17881-2f4f-4655-83df-8979581837e6" (UID: "23d17881-2f4f-4655-83df-8979581837e6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:46:19.789641 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.789620 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d17881-2f4f-4655-83df-8979581837e6-kube-api-access-4vvw5" (OuterVolumeSpecName: "kube-api-access-4vvw5") pod "23d17881-2f4f-4655-83df-8979581837e6" (UID: "23d17881-2f4f-4655-83df-8979581837e6"). InnerVolumeSpecName "kube-api-access-4vvw5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:46:19.794153 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.794123 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-util" (OuterVolumeSpecName: "util") pod "23d17881-2f4f-4655-83df-8979581837e6" (UID: "23d17881-2f4f-4655-83df-8979581837e6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:46:19.888999 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.888908 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:19.888999 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.888939 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23d17881-2f4f-4655-83df-8979581837e6-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:19.888999 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:19.888949 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vvw5\" (UniqueName: \"kubernetes.io/projected/23d17881-2f4f-4655-83df-8979581837e6-kube-api-access-4vvw5\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:46:20.608757 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:20.608725 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" Jun 10 12:46:20.608757 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:20.608739 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5" event={"ID":"23d17881-2f4f-4655-83df-8979581837e6","Type":"ContainerDied","Data":"32342560b175a4a97a375ea8bebf482597c4bab02c11202611d9e833177fc875"} Jun 10 12:46:20.609218 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:20.608787 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32342560b175a4a97a375ea8bebf482597c4bab02c11202611d9e833177fc875" Jun 10 12:46:28.290459 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:28.290413 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:32.370743 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:32.370712 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:33.406673 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:33.406609 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:37.594892 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:37.594860 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:40.476119 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:40.476068 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:40.581949 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:40.581903 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:43.084203 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:43.084166 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:43.604791 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:43.604759 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:44.977998 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:44.977965 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:46.327540 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327507 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9"] Jun 10 12:46:46.327921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327868 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerName="extract" Jun 10 12:46:46.327921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327881 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerName="extract" Jun 10 12:46:46.327921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327891 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerName="pull" Jun 10 12:46:46.327921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327897 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerName="pull" Jun 10 12:46:46.327921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327905 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerName="extract" Jun 10 12:46:46.327921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327911 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerName="extract" Jun 10 12:46:46.327921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327917 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="527361d7-e4c6-4f05-90b1-560e11228c77" containerName="util" Jun 10 12:46:46.327921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327921 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="527361d7-e4c6-4f05-90b1-560e11228c77" containerName="util" Jun 10 12:46:46.327921 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327926 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerName="pull" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327931 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerName="pull" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327937 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d17881-2f4f-4655-83df-8979581837e6" containerName="pull" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327941 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d17881-2f4f-4655-83df-8979581837e6" containerName="pull" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327947 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d17881-2f4f-4655-83df-8979581837e6" containerName="extract" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327951 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d17881-2f4f-4655-83df-8979581837e6" containerName="extract" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327963 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d17881-2f4f-4655-83df-8979581837e6" containerName="util" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327970 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d17881-2f4f-4655-83df-8979581837e6" containerName="util" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327978 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="527361d7-e4c6-4f05-90b1-560e11228c77" containerName="extract" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327983 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="527361d7-e4c6-4f05-90b1-560e11228c77" containerName="extract" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327988 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerName="util" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327993 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerName="util" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.327997 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="527361d7-e4c6-4f05-90b1-560e11228c77" containerName="pull" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.328002 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="527361d7-e4c6-4f05-90b1-560e11228c77" containerName="pull" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.328011 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerName="util" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.328016 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerName="util" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.328068 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="527361d7-e4c6-4f05-90b1-560e11228c77" containerName="extract" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.328075 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="03d63aba-abc2-4dc1-933c-71f1d0cb29f7" containerName="extract" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.328081 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1c12211-3534-4788-b5bd-39a0be5843fd" containerName="extract" Jun 10 12:46:46.328237 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.328090 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="23d17881-2f4f-4655-83df-8979581837e6" containerName="extract" Jun 10 12:46:46.331223 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.331207 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:46:46.338117 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.338078 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 10 12:46:46.339426 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.339403 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7fzms\"" Jun 10 12:46:46.339540 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.339437 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 10 12:46:46.361060 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.361032 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9"] Jun 10 12:46:46.428509 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.428465 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkbpz\" (UniqueName: \"kubernetes.io/projected/8b11f49a-ce5d-4587-9e6c-7b23455d9f01-kube-api-access-lkbpz\") pod \"limitador-operator-controller-manager-859d7b78bd-gzqh9\" (UID: \"8b11f49a-ce5d-4587-9e6c-7b23455d9f01\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:46:46.529988 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.529944 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkbpz\" (UniqueName: \"kubernetes.io/projected/8b11f49a-ce5d-4587-9e6c-7b23455d9f01-kube-api-access-lkbpz\") pod \"limitador-operator-controller-manager-859d7b78bd-gzqh9\" (UID: \"8b11f49a-ce5d-4587-9e6c-7b23455d9f01\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:46:46.549653 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.549618 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkbpz\" (UniqueName: \"kubernetes.io/projected/8b11f49a-ce5d-4587-9e6c-7b23455d9f01-kube-api-access-lkbpz\") pod \"limitador-operator-controller-manager-859d7b78bd-gzqh9\" (UID: \"8b11f49a-ce5d-4587-9e6c-7b23455d9f01\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:46:46.643005 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.642915 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:46:46.796626 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.796594 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9"] Jun 10 12:46:46.798874 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:46:46.798848 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b11f49a_ce5d_4587_9e6c_7b23455d9f01.slice/crio-072b44fb6acd0e0737bfed2acbaacb3ba424e2ea009e3bf77fc19f4199126c3f WatchSource:0}: Error finding container 072b44fb6acd0e0737bfed2acbaacb3ba424e2ea009e3bf77fc19f4199126c3f: Status 404 returned error can't find the container with id 072b44fb6acd0e0737bfed2acbaacb3ba424e2ea009e3bf77fc19f4199126c3f Jun 10 12:46:46.890675 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:46.890633 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:47.768178 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:47.768142 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" event={"ID":"8b11f49a-ce5d-4587-9e6c-7b23455d9f01","Type":"ContainerStarted","Data":"072b44fb6acd0e0737bfed2acbaacb3ba424e2ea009e3bf77fc19f4199126c3f"} Jun 10 12:46:48.390061 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:48.390028 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:48.493657 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:48.493621 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:49.782876 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:49.782844 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" event={"ID":"8b11f49a-ce5d-4587-9e6c-7b23455d9f01","Type":"ContainerStarted","Data":"46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3"} Jun 10 12:46:49.783286 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:49.782903 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:46:49.808404 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:49.808356 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" podStartSLOduration=1.5995678359999999 podStartE2EDuration="3.808340348s" totalImagesPullingTime="2.208772512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:46:49.805071428 +0000 UTC m=+600.015917414" watchObservedRunningTime="2026-06-10 12:46:49.808340348 +0000 UTC m=+600.019186336" Jun 10 12:46:50.246822 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:50.246794 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:46:50.247023 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:50.246954 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:46:52.946609 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:52.946570 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:53.086829 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:53.086781 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:53.192671 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:53.192629 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:53.295406 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:53.295279 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:53.404223 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:53.404192 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:58.081001 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:58.080965 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:58.180287 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:58.180229 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:46:58.292314 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:46:58.292283 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:00.791407 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:00.791325 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:47:02.975336 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:02.975305 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:03.104038 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.103970 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:03.589507 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.589473 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9"] Jun 10 12:47:03.589725 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.589698 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" podUID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" containerName="manager" containerID="cri-o://46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3" gracePeriod=2 Jun 10 12:47:03.612483 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.612449 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9"] Jun 10 12:47:03.616961 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:47:03.616917 2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8b11f49a-ce5d-4587-9e6c-7b23455d9f01\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-gzqh9\": pods \"limitador-operator-controller-manager-859d7b78bd-gzqh9\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:47:03.652163 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.652132 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8"] Jun 10 12:47:03.652528 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.652516 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" containerName="manager" Jun 10 12:47:03.652575 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.652530 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" containerName="manager" Jun 10 12:47:03.652610 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.652585 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" containerName="manager" Jun 10 12:47:03.655817 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.655797 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" Jun 10 12:47:03.659942 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:47:03.659910 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gzqh9\" is forbidden: User \"system:node:ip-10-0-138-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-99.ec2.internal' and this object" podUID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:47:03.676741 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.676712 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8"] Jun 10 12:47:03.787696 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.787665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qthqz\" (UniqueName: \"kubernetes.io/projected/30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4-kube-api-access-qthqz\") pod \"limitador-operator-controller-manager-859d7b78bd-vnbs8\" (UID: \"30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" Jun 10 12:47:03.850264 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.850196 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:47:03.854436 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:47:03.854409 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gzqh9\" is forbidden: User \"system:node:ip-10-0-138-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-99.ec2.internal' and this object" podUID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:47:03.863985 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.863952 2582 generic.go:356] "Generic (PLEG): container finished" podID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" containerID="46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3" exitCode=0 Jun 10 12:47:03.864173 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.864009 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:47:03.864173 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.864060 2582 scope.go:117] "RemoveContainer" containerID="46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3" Jun 10 12:47:03.866956 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:47:03.866925 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gzqh9\" is forbidden: User \"system:node:ip-10-0-138-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-99.ec2.internal' and this object" podUID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:47:03.875191 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.875172 2582 scope.go:117] "RemoveContainer" containerID="46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3" Jun 10 12:47:03.875505 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:47:03.875484 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3\": container with ID starting with 46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3 not found: ID does not exist" containerID="46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3" Jun 10 12:47:03.875562 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.875515 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3"} err="failed to get container status \"46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3\": rpc error: code = NotFound desc = could not find container \"46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3\": container with ID starting with 46a1f6ed55358b07bc1b5e1f570a1c63b9bbabbcd46f5292fbc20330961dcec3 not found: ID does not exist" Jun 10 12:47:03.888861 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.888839 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkbpz\" (UniqueName: \"kubernetes.io/projected/8b11f49a-ce5d-4587-9e6c-7b23455d9f01-kube-api-access-lkbpz\") pod \"8b11f49a-ce5d-4587-9e6c-7b23455d9f01\" (UID: \"8b11f49a-ce5d-4587-9e6c-7b23455d9f01\") " Jun 10 12:47:03.889012 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.888997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qthqz\" (UniqueName: \"kubernetes.io/projected/30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4-kube-api-access-qthqz\") pod \"limitador-operator-controller-manager-859d7b78bd-vnbs8\" (UID: \"30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" Jun 10 12:47:03.891066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.891033 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b11f49a-ce5d-4587-9e6c-7b23455d9f01-kube-api-access-lkbpz" (OuterVolumeSpecName: "kube-api-access-lkbpz") pod "8b11f49a-ce5d-4587-9e6c-7b23455d9f01" (UID: "8b11f49a-ce5d-4587-9e6c-7b23455d9f01"). InnerVolumeSpecName "kube-api-access-lkbpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:47:03.912766 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.912741 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qthqz\" (UniqueName: \"kubernetes.io/projected/30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4-kube-api-access-qthqz\") pod \"limitador-operator-controller-manager-859d7b78bd-vnbs8\" (UID: \"30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" Jun 10 12:47:03.975565 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.975529 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" Jun 10 12:47:03.989479 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:03.989457 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkbpz\" (UniqueName: \"kubernetes.io/projected/8b11f49a-ce5d-4587-9e6c-7b23455d9f01-kube-api-access-lkbpz\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:47:04.133536 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:04.133508 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8"] Jun 10 12:47:04.134811 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:47:04.134784 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c5fe3c_9ddd_4b40_93a0_09b3baa64bc4.slice/crio-a838a9d4ba75dd5186cac4058ce3ee36aa06c62151960b9cc6af34614541d1a8 WatchSource:0}: Error finding container a838a9d4ba75dd5186cac4058ce3ee36aa06c62151960b9cc6af34614541d1a8: Status 404 returned error can't find the container with id a838a9d4ba75dd5186cac4058ce3ee36aa06c62151960b9cc6af34614541d1a8 Jun 10 12:47:04.175199 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:47:04.175164 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gzqh9\" is forbidden: User \"system:node:ip-10-0-138-99.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-99.ec2.internal' and this object" podUID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gzqh9" Jun 10 12:47:04.257582 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:04.257541 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b11f49a-ce5d-4587-9e6c-7b23455d9f01" path="/var/lib/kubelet/pods/8b11f49a-ce5d-4587-9e6c-7b23455d9f01/volumes" Jun 10 12:47:04.440272 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:04.440186 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:04.576325 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:04.576290 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:04.872426 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:04.872389 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" event={"ID":"30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4","Type":"ContainerStarted","Data":"48b48372f3e8bb8bdd1306297a82562e058ebe9ca47f643f7df2a1fa2ba88ee7"} Jun 10 12:47:04.872603 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:04.872449 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" event={"ID":"30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4","Type":"ContainerStarted","Data":"a838a9d4ba75dd5186cac4058ce3ee36aa06c62151960b9cc6af34614541d1a8"} Jun 10 12:47:04.872603 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:04.872537 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" Jun 10 12:47:04.897632 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:04.897576 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" podStartSLOduration=1.897561288 podStartE2EDuration="1.897561288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:47:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:47:04.895691893 +0000 UTC m=+615.106537893" watchObservedRunningTime="2026-06-10 12:47:04.897561288 +0000 UTC m=+615.108407275" Jun 10 12:47:05.069855 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:05.069822 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:05.202680 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:05.202598 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:07.004824 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:07.004795 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:07.129479 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:07.129425 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:11.718733 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:11.718684 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:11.822826 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:11.822794 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:11.908396 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:11.908362 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:15.260066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:15.260034 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:15.418241 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:15.418210 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:15.880671 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:15.880643 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnbs8" Jun 10 12:47:22.281208 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:22.281171 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:22.378472 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:22.378416 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:24.892413 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:24.892384 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:24.995351 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:24.995318 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:30.291748 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:30.291707 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:30.382577 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:30.382546 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:30.873576 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:30.873549 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:30.972611 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:30.972581 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:33.903864 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:33.903829 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:33.978286 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:33.978249 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:34.480512 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:34.480478 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:34.577452 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:34.577423 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:37.621783 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:37.621754 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:38.274634 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:38.274603 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:38.376926 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:38.376890 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:38.495245 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:38.495211 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:38.996505 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:38.996471 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:47:39.000183 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.000164 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:39.003602 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.003577 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 10 12:47:39.003707 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.003587 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5qnsx\"" Jun 10 12:47:39.004532 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.004438 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhh6m\" (UniqueName: \"kubernetes.io/projected/d904d244-e090-43bc-be52-99409a02790f-kube-api-access-nhh6m\") pod \"limitador-limitador-54d6d988d5-srx49\" (UID: \"d904d244-e090-43bc-be52-99409a02790f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:39.004532 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.004507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d904d244-e090-43bc-be52-99409a02790f-config-file\") pod \"limitador-limitador-54d6d988d5-srx49\" (UID: \"d904d244-e090-43bc-be52-99409a02790f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:39.011016 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.010990 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:47:39.033671 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.033637 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:39.041112 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.041075 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:47:39.101832 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.101800 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:39.105106 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.105074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d904d244-e090-43bc-be52-99409a02790f-config-file\") pod \"limitador-limitador-54d6d988d5-srx49\" (UID: \"d904d244-e090-43bc-be52-99409a02790f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:39.105190 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.105177 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhh6m\" (UniqueName: \"kubernetes.io/projected/d904d244-e090-43bc-be52-99409a02790f-kube-api-access-nhh6m\") pod \"limitador-limitador-54d6d988d5-srx49\" (UID: \"d904d244-e090-43bc-be52-99409a02790f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:39.105709 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.105692 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d904d244-e090-43bc-be52-99409a02790f-config-file\") pod \"limitador-limitador-54d6d988d5-srx49\" (UID: \"d904d244-e090-43bc-be52-99409a02790f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:39.113940 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.113917 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhh6m\" (UniqueName: \"kubernetes.io/projected/d904d244-e090-43bc-be52-99409a02790f-kube-api-access-nhh6m\") pod \"limitador-limitador-54d6d988d5-srx49\" (UID: \"d904d244-e090-43bc-be52-99409a02790f\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:39.312716 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.312685 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:39.455064 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.455038 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:47:39.456535 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:47:39.456506 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd904d244_e090_43bc_be52_99409a02790f.slice/crio-a9cd954beb22914d7a9e9672cedd70ce803ec0f6c0d203f8e876f1635ccb703e WatchSource:0}: Error finding container a9cd954beb22914d7a9e9672cedd70ce803ec0f6c0d203f8e876f1635ccb703e: Status 404 returned error can't find the container with id a9cd954beb22914d7a9e9672cedd70ce803ec0f6c0d203f8e876f1635ccb703e Jun 10 12:47:39.458514 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:39.458494 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 12:47:40.082855 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:40.082812 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" event={"ID":"d904d244-e090-43bc-be52-99409a02790f","Type":"ContainerStarted","Data":"a9cd954beb22914d7a9e9672cedd70ce803ec0f6c0d203f8e876f1635ccb703e"} Jun 10 12:47:41.106331 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:41.106284 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:41.208328 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:41.208290 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:42.096812 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:42.096783 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" event={"ID":"d904d244-e090-43bc-be52-99409a02790f","Type":"ContainerStarted","Data":"f4ee9234095768218539ebe7203dc650d2b64fcf77a6d4379f8cadd0a3749914"} Jun 10 12:47:42.096984 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:42.096904 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:42.119454 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:42.119402 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" podStartSLOduration=1.5738931790000001 podStartE2EDuration="4.119386251s" totalImagesPullingTime="2.545493072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:47:42.116764837 +0000 UTC m=+652.327610825" watchObservedRunningTime="2026-06-10 12:47:42.119386251 +0000 UTC m=+652.330232250" Jun 10 12:47:53.104020 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:53.103991 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-srx49" Jun 10 12:47:59.268013 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:59.267979 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:59.361551 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:59.361505 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:47:59.685663 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:47:59.685630 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:48:10.307508 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:10.307464 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:48:10.408747 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:10.408714 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:48:10.580704 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:10.580609 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:48:11.874922 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:11.874885 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9"] Jun 10 12:48:11.881627 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:11.881605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:11.884867 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:11.884850 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 10 12:48:11.884967 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:11.884854 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 10 12:48:11.886595 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:11.886563 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhnkq\"" Jun 10 12:48:11.890753 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:11.890729 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9"] Jun 10 12:48:11.998755 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:11.998717 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:11.998946 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:11.998788 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:11.998946 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:11.998820 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r98n8\" (UniqueName: \"kubernetes.io/projected/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-kube-api-access-r98n8\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:12.099341 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:12.099309 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r98n8\" (UniqueName: \"kubernetes.io/projected/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-kube-api-access-r98n8\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:12.099529 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:12.099411 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:12.099529 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:12.099475 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:12.099768 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:12.099740 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:12.099864 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:12.099807 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:12.109330 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:12.109302 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r98n8\" (UniqueName: \"kubernetes.io/projected/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-kube-api-access-r98n8\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:12.192376 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:12.192296 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:12.342557 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:12.342485 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9"] Jun 10 12:48:12.344993 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:48:12.344967 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d98aec7_ef89_4e9f_a2b1_09650b2ff1ef.slice/crio-4648df019efada912672acf5d064ccf8d18a1c3d4af5a01ae29fdeac2392fcd0 WatchSource:0}: Error finding container 4648df019efada912672acf5d064ccf8d18a1c3d4af5a01ae29fdeac2392fcd0: Status 404 returned error can't find the container with id 4648df019efada912672acf5d064ccf8d18a1c3d4af5a01ae29fdeac2392fcd0 Jun 10 12:48:13.278951 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:13.278916 2582 generic.go:356] "Generic (PLEG): container finished" podID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerID="ebc35cb735b0f1488ce09682de534d7bf7ea448a90b444fcfc6185362ff041a6" exitCode=0 Jun 10 12:48:13.279426 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:13.279010 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" event={"ID":"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef","Type":"ContainerDied","Data":"ebc35cb735b0f1488ce09682de534d7bf7ea448a90b444fcfc6185362ff041a6"} Jun 10 12:48:13.279426 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:13.279060 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" event={"ID":"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef","Type":"ContainerStarted","Data":"4648df019efada912672acf5d064ccf8d18a1c3d4af5a01ae29fdeac2392fcd0"} Jun 10 12:48:14.291660 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:14.291568 2582 generic.go:356] "Generic (PLEG): container finished" podID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerID="560673c7d1cd35a8994748cd2bc08ab9f600e50ab5868d0828cfb84668902ac9" exitCode=0 Jun 10 12:48:14.291660 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:14.291630 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" event={"ID":"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef","Type":"ContainerDied","Data":"560673c7d1cd35a8994748cd2bc08ab9f600e50ab5868d0828cfb84668902ac9"} Jun 10 12:48:15.300347 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:15.300306 2582 generic.go:356] "Generic (PLEG): container finished" podID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerID="4af142ceb82aca27ab26c4adb7f91df035fae702696b96dc0e73d19525df77ba" exitCode=0 Jun 10 12:48:15.300784 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:15.300383 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" event={"ID":"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef","Type":"ContainerDied","Data":"4af142ceb82aca27ab26c4adb7f91df035fae702696b96dc0e73d19525df77ba"} Jun 10 12:48:16.463429 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.463399 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:16.542044 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.542007 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-bundle\") pod \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " Jun 10 12:48:16.542271 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.542142 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-util\") pod \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " Jun 10 12:48:16.542271 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.542198 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r98n8\" (UniqueName: \"kubernetes.io/projected/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-kube-api-access-r98n8\") pod \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\" (UID: \"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef\") " Jun 10 12:48:16.542539 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.542515 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-bundle" (OuterVolumeSpecName: "bundle") pod "9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" (UID: "9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:48:16.544439 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.544415 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-kube-api-access-r98n8" (OuterVolumeSpecName: "kube-api-access-r98n8") pod "9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" (UID: "9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef"). InnerVolumeSpecName "kube-api-access-r98n8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:48:16.547987 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.547953 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-util" (OuterVolumeSpecName: "util") pod "9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" (UID: "9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:48:16.643407 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.643308 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r98n8\" (UniqueName: \"kubernetes.io/projected/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-kube-api-access-r98n8\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:48:16.643407 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.643347 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:48:16.643407 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:16.643358 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef-util\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:48:17.318765 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:17.318742 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" Jun 10 12:48:17.318933 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:17.318736 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74844vf9" event={"ID":"9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef","Type":"ContainerDied","Data":"4648df019efada912672acf5d064ccf8d18a1c3d4af5a01ae29fdeac2392fcd0"} Jun 10 12:48:17.318933 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:17.318832 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4648df019efada912672acf5d064ccf8d18a1c3d4af5a01ae29fdeac2392fcd0" Jun 10 12:48:31.351957 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:31.351913 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:48:35.757434 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:35.757404 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:48:35.870605 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:48:35.870571 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:06.356018 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:06.355985 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:06.686189 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:06.686116 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:11.912450 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:11.912406 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:12.060812 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:12.060779 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:12.140711 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:12.140681 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:15.426783 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:15.426749 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:15.529409 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:15.529367 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:41.523625 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:41.523590 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:44.956743 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:44.956683 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:45.053550 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:45.053497 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:45.713269 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:45.713230 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:45.944285 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:45.944247 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:46.204515 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.204471 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:46.305599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.305570 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:46.336149 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.336115 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-ss2qm"] Jun 10 12:49:46.336598 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.336580 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerName="pull" Jun 10 12:49:46.336644 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.336603 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerName="pull" Jun 10 12:49:46.336644 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.336628 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerName="util" Jun 10 12:49:46.336644 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.336634 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerName="util" Jun 10 12:49:46.336644 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.336640 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerName="extract" Jun 10 12:49:46.336813 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.336647 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerName="extract" Jun 10 12:49:46.336813 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.336725 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d98aec7-ef89-4e9f-a2b1-09650b2ff1ef" containerName="extract" Jun 10 12:49:46.339970 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.339950 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" Jun 10 12:49:46.342858 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.342839 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 10 12:49:46.342951 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.342923 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vtnkk\"" Jun 10 12:49:46.355935 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.355912 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-ss2qm"] Jun 10 12:49:46.362140 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.360630 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:49:46.478280 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.478197 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmg54\" (UniqueName: \"kubernetes.io/projected/13395be0-f360-4121-b1a4-293a6aec2163-kube-api-access-bmg54\") pod \"payload-processing-699b6cd6c-ss2qm\" (UID: \"13395be0-f360-4121-b1a4-293a6aec2163\") " pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" Jun 10 12:49:46.579764 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.579724 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmg54\" (UniqueName: \"kubernetes.io/projected/13395be0-f360-4121-b1a4-293a6aec2163-kube-api-access-bmg54\") pod \"payload-processing-699b6cd6c-ss2qm\" (UID: \"13395be0-f360-4121-b1a4-293a6aec2163\") " pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" Jun 10 12:49:46.588681 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.588655 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmg54\" (UniqueName: \"kubernetes.io/projected/13395be0-f360-4121-b1a4-293a6aec2163-kube-api-access-bmg54\") pod \"payload-processing-699b6cd6c-ss2qm\" (UID: \"13395be0-f360-4121-b1a4-293a6aec2163\") " pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" Jun 10 12:49:46.650046 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.650008 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" Jun 10 12:49:46.805304 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.805273 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-ss2qm"] Jun 10 12:49:46.806743 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:49:46.806715 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13395be0_f360_4121_b1a4_293a6aec2163.slice/crio-bf2f8cce88c5b6b6643ecb6c28c85167517cef3085333fcced3fd99c062a7736 WatchSource:0}: Error finding container bf2f8cce88c5b6b6643ecb6c28c85167517cef3085333fcced3fd99c062a7736: Status 404 returned error can't find the container with id bf2f8cce88c5b6b6643ecb6c28c85167517cef3085333fcced3fd99c062a7736 Jun 10 12:49:46.849788 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:46.849751 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" event={"ID":"13395be0-f360-4121-b1a4-293a6aec2163","Type":"ContainerStarted","Data":"bf2f8cce88c5b6b6643ecb6c28c85167517cef3085333fcced3fd99c062a7736"} Jun 10 12:49:49.873366 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:49.873332 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" event={"ID":"13395be0-f360-4121-b1a4-293a6aec2163","Type":"ContainerStarted","Data":"a8f9b3277c47590225fabcde240a0e0175078ef3b7c796b6691dbf7e9fe158a2"} Jun 10 12:49:49.873751 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:49.873459 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" Jun 10 12:49:49.894652 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:49:49.894600 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" podStartSLOduration=1.315927264 podStartE2EDuration="3.894584975s" totalImagesPullingTime="2.578657711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:49.893223517 +0000 UTC m=+780.104069504" watchObservedRunningTime="2026-06-10 12:49:49.894584975 +0000 UTC m=+780.105430990" Jun 10 12:50:00.880938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:00.880862 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-ss2qm" Jun 10 12:50:35.916755 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:35.916716 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:36.015811 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:36.015780 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:36.500606 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:36.500571 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:44.467158 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:44.467079 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:44.542391 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:44.542351 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:48.784515 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:48.784445 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:49.212895 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:49.212856 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:50:54.416731 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:54.416701 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:54.574554 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:50:54.574522 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:51:03.442770 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.442743 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2"] Jun 10 12:51:03.446496 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.446477 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.451075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.451050 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nn99j\"" Jun 10 12:51:03.451075 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.451075 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 10 12:51:03.451293 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.451060 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 10 12:51:03.451293 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.451051 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 10 12:51:03.460515 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.460487 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2"] Jun 10 12:51:03.486036 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.485992 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:51:03.576909 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.576874 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.577153 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.576917 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29b02545-4a49-48fd-ba87-d09b27252bde-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.577153 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.576953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrrvw\" (UniqueName: \"kubernetes.io/projected/29b02545-4a49-48fd-ba87-d09b27252bde-kube-api-access-jrrvw\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.577153 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.577019 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.577153 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.577065 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.577153 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.577087 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.677966 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.677921 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.678178 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.677973 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.678244 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.678165 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.678309 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.678292 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.678364 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.678323 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29b02545-4a49-48fd-ba87-d09b27252bde-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.678424 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.678360 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrrvw\" (UniqueName: \"kubernetes.io/projected/29b02545-4a49-48fd-ba87-d09b27252bde-kube-api-access-jrrvw\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.678424 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.678414 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.678526 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.678495 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.678688 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.678662 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.680572 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.680543 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29b02545-4a49-48fd-ba87-d09b27252bde-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.680810 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.680791 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29b02545-4a49-48fd-ba87-d09b27252bde-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.687142 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.687074 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrrvw\" (UniqueName: \"kubernetes.io/projected/29b02545-4a49-48fd-ba87-d09b27252bde-kube-api-access-jrrvw\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-df8g2\" (UID: \"29b02545-4a49-48fd-ba87-d09b27252bde\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.758352 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.758260 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:03.882755 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.882719 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:51:03.924256 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:51:03.924211 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b02545_4a49_48fd_ba87_d09b27252bde.slice/crio-564f1310d8a73f0f5e64d52ee73f20d36a6278c96189ba1a59e020042986e3c3 WatchSource:0}: Error finding container 564f1310d8a73f0f5e64d52ee73f20d36a6278c96189ba1a59e020042986e3c3: Status 404 returned error can't find the container with id 564f1310d8a73f0f5e64d52ee73f20d36a6278c96189ba1a59e020042986e3c3 Jun 10 12:51:03.924628 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:03.924606 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2"] Jun 10 12:51:04.315612 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:04.315576 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" event={"ID":"29b02545-4a49-48fd-ba87-d09b27252bde","Type":"ContainerStarted","Data":"564f1310d8a73f0f5e64d52ee73f20d36a6278c96189ba1a59e020042986e3c3"} Jun 10 12:51:07.336583 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:07.336535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" event={"ID":"29b02545-4a49-48fd-ba87-d09b27252bde","Type":"ContainerStarted","Data":"c8d77ce307b925c0da40859c8d8b75512ec32d6c0af8029fae30d6411b5b1094"} Jun 10 12:51:09.350707 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:09.350676 2582 generic.go:356] "Generic (PLEG): container finished" podID="29b02545-4a49-48fd-ba87-d09b27252bde" containerID="c8d77ce307b925c0da40859c8d8b75512ec32d6c0af8029fae30d6411b5b1094" exitCode=0 Jun 10 12:51:09.351084 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:09.350714 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" event={"ID":"29b02545-4a49-48fd-ba87-d09b27252bde","Type":"ContainerDied","Data":"c8d77ce307b925c0da40859c8d8b75512ec32d6c0af8029fae30d6411b5b1094"} Jun 10 12:51:11.365397 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:11.365361 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" event={"ID":"29b02545-4a49-48fd-ba87-d09b27252bde","Type":"ContainerStarted","Data":"11e6caff1439af55b6fe2e7276b5ba5c3f0ee3adf7998cad6a8e2b5ef69ee6f0"} Jun 10 12:51:11.393660 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:11.393607 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" podStartSLOduration=2.72301264 podStartE2EDuration="8.393590566s" totalImagesPullingTime="4.40355721s" totalInitContainerRuntime="1.267020716s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:51:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:51:11.392260166 +0000 UTC m=+861.603106152" watchObservedRunningTime="2026-06-10 12:51:11.393590566 +0000 UTC m=+861.604436606" Jun 10 12:51:13.758834 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:13.758792 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:13.759264 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:13.758886 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:13.771793 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:13.771767 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:14.395034 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:14.395006 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-df8g2" Jun 10 12:51:17.952327 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:17.952299 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:51:18.669820 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:18.669783 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:51:18.929177 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:18.929057 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:51:37.430171 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:37.430128 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:51:37.976717 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:37.976680 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:51:43.441826 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:43.441789 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:51:44.874689 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:44.874658 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:51:45.582349 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:45.582316 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:51:50.288115 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:50.288072 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:51:50.290159 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:51:50.290137 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:52:00.770233 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.770202 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6996bff5fb-d4j5m"] Jun 10 12:52:00.773792 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.773776 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:00.805062 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.805030 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6996bff5fb-d4j5m"] Jun 10 12:52:00.844497 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.844463 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:52:00.931848 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.931815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/723aa781-0e4d-413d-8f8f-65023f6c378b-console-oauth-config\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:00.931848 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.931851 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-console-config\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:00.932050 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.931877 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-oauth-serving-cert\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:00.932050 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.931986 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-trusted-ca-bundle\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:00.932050 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.932003 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-service-ca\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:00.932050 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.932018 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pg6c\" (UniqueName: \"kubernetes.io/projected/723aa781-0e4d-413d-8f8f-65023f6c378b-kube-api-access-8pg6c\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:00.932203 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.932064 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/723aa781-0e4d-413d-8f8f-65023f6c378b-console-serving-cert\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:00.949317 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:00.949285 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:52:01.032790 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.032682 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/723aa781-0e4d-413d-8f8f-65023f6c378b-console-oauth-config\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.032790 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.032731 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-console-config\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.032790 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.032764 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-oauth-serving-cert\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.033078 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.032854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-trusted-ca-bundle\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.033078 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.032880 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-service-ca\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.033078 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.032905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pg6c\" (UniqueName: \"kubernetes.io/projected/723aa781-0e4d-413d-8f8f-65023f6c378b-kube-api-access-8pg6c\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.033078 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.032962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/723aa781-0e4d-413d-8f8f-65023f6c378b-console-serving-cert\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.033688 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.033658 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-console-config\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.033799 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.033764 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-service-ca\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.033879 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.033855 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-oauth-serving-cert\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.033925 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.033861 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723aa781-0e4d-413d-8f8f-65023f6c378b-trusted-ca-bundle\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.035415 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.035395 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/723aa781-0e4d-413d-8f8f-65023f6c378b-console-oauth-config\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.035553 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.035536 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/723aa781-0e4d-413d-8f8f-65023f6c378b-console-serving-cert\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.043825 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.043796 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pg6c\" (UniqueName: \"kubernetes.io/projected/723aa781-0e4d-413d-8f8f-65023f6c378b-kube-api-access-8pg6c\") pod \"console-6996bff5fb-d4j5m\" (UID: \"723aa781-0e4d-413d-8f8f-65023f6c378b\") " pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.083680 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.083635 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:01.233412 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.233381 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6996bff5fb-d4j5m"] Jun 10 12:52:01.234974 ip-10-0-138-99 kubenswrapper[2582]: W0610 12:52:01.234950 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723aa781_0e4d_413d_8f8f_65023f6c378b.slice/crio-ea5cbf82f7d63919b906d9b05c5208217114c890b75826b3e76695e204245a69 WatchSource:0}: Error finding container ea5cbf82f7d63919b906d9b05c5208217114c890b75826b3e76695e204245a69: Status 404 returned error can't find the container with id ea5cbf82f7d63919b906d9b05c5208217114c890b75826b3e76695e204245a69 Jun 10 12:52:01.674136 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.674077 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6996bff5fb-d4j5m" event={"ID":"723aa781-0e4d-413d-8f8f-65023f6c378b","Type":"ContainerStarted","Data":"ff4fb52d9ec19653c61d6d65015c0e5c7859cfb44384bfffae8b6bc6a6da3620"} Jun 10 12:52:01.674287 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:01.674149 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6996bff5fb-d4j5m" event={"ID":"723aa781-0e4d-413d-8f8f-65023f6c378b","Type":"ContainerStarted","Data":"ea5cbf82f7d63919b906d9b05c5208217114c890b75826b3e76695e204245a69"} Jun 10 12:52:04.310849 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:04.310804 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:52:05.308564 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:05.308501 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6996bff5fb-d4j5m" podStartSLOduration=5.308482085 podStartE2EDuration="5.308482085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:52:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:52:01.706373436 +0000 UTC m=+911.917219423" watchObservedRunningTime="2026-06-10 12:52:05.308482085 +0000 UTC m=+915.519328079" Jun 10 12:52:05.308937 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:05.308915 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-srx49"] Jun 10 12:52:11.084833 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:11.084778 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:11.085504 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:11.084862 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:11.090034 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:11.090009 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:11.741833 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:11.741804 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6996bff5fb-d4j5m" Jun 10 12:52:11.819455 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:11.819413 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7756944cf8-cvd5q"] Jun 10 12:52:11.980308 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:11.980265 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:52:12.116944 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:12.116906 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:52:12.211332 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:12.211299 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:52:36.841309 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:36.841237 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7756944cf8-cvd5q" podUID="ec616f84-5301-4afa-a2c9-df4154367be8" containerName="console" containerID="cri-o://3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6" gracePeriod=15 Jun 10 12:52:37.100535 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.100466 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:52:37.261522 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.261486 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-oauth-serving-cert\") pod \"ec616f84-5301-4afa-a2c9-df4154367be8\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " Jun 10 12:52:37.261689 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.261565 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-trusted-ca-bundle\") pod \"ec616f84-5301-4afa-a2c9-df4154367be8\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " Jun 10 12:52:37.261689 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.261595 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-oauth-config\") pod \"ec616f84-5301-4afa-a2c9-df4154367be8\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " Jun 10 12:52:37.261689 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.261619 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-console-config\") pod \"ec616f84-5301-4afa-a2c9-df4154367be8\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " Jun 10 12:52:37.261689 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.261656 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bxkz\" (UniqueName: \"kubernetes.io/projected/ec616f84-5301-4afa-a2c9-df4154367be8-kube-api-access-7bxkz\") pod \"ec616f84-5301-4afa-a2c9-df4154367be8\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " Jun 10 12:52:37.261925 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.261724 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-serving-cert\") pod \"ec616f84-5301-4afa-a2c9-df4154367be8\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " Jun 10 12:52:37.261925 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.261774 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-service-ca\") pod \"ec616f84-5301-4afa-a2c9-df4154367be8\" (UID: \"ec616f84-5301-4afa-a2c9-df4154367be8\") " Jun 10 12:52:37.262079 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.262048 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ec616f84-5301-4afa-a2c9-df4154367be8" (UID: "ec616f84-5301-4afa-a2c9-df4154367be8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:52:37.262190 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.262170 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-console-config" (OuterVolumeSpecName: "console-config") pod "ec616f84-5301-4afa-a2c9-df4154367be8" (UID: "ec616f84-5301-4afa-a2c9-df4154367be8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:52:37.262234 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.262195 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ec616f84-5301-4afa-a2c9-df4154367be8" (UID: "ec616f84-5301-4afa-a2c9-df4154367be8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:52:37.262387 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.262366 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-service-ca" (OuterVolumeSpecName: "service-ca") pod "ec616f84-5301-4afa-a2c9-df4154367be8" (UID: "ec616f84-5301-4afa-a2c9-df4154367be8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:52:37.264066 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.264031 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ec616f84-5301-4afa-a2c9-df4154367be8" (UID: "ec616f84-5301-4afa-a2c9-df4154367be8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:52:37.264185 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.264061 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ec616f84-5301-4afa-a2c9-df4154367be8" (UID: "ec616f84-5301-4afa-a2c9-df4154367be8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:52:37.264185 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.264077 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec616f84-5301-4afa-a2c9-df4154367be8-kube-api-access-7bxkz" (OuterVolumeSpecName: "kube-api-access-7bxkz") pod "ec616f84-5301-4afa-a2c9-df4154367be8" (UID: "ec616f84-5301-4afa-a2c9-df4154367be8"). InnerVolumeSpecName "kube-api-access-7bxkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:52:37.363080 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.362995 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-oauth-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:52:37.363080 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.363025 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-trusted-ca-bundle\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:52:37.363080 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.363035 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-oauth-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:52:37.363080 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.363043 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-console-config\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:52:37.363080 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.363052 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bxkz\" (UniqueName: \"kubernetes.io/projected/ec616f84-5301-4afa-a2c9-df4154367be8-kube-api-access-7bxkz\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:52:37.363080 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.363062 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec616f84-5301-4afa-a2c9-df4154367be8-console-serving-cert\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:52:37.363080 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.363070 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec616f84-5301-4afa-a2c9-df4154367be8-service-ca\") on node \"ip-10-0-138-99.ec2.internal\" DevicePath \"\"" Jun 10 12:52:37.894247 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.894211 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec616f84-5301-4afa-a2c9-df4154367be8" containerID="3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6" exitCode=0 Jun 10 12:52:37.894722 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.894262 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7756944cf8-cvd5q" Jun 10 12:52:37.894722 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.894292 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7756944cf8-cvd5q" event={"ID":"ec616f84-5301-4afa-a2c9-df4154367be8","Type":"ContainerDied","Data":"3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6"} Jun 10 12:52:37.894722 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.894339 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7756944cf8-cvd5q" event={"ID":"ec616f84-5301-4afa-a2c9-df4154367be8","Type":"ContainerDied","Data":"db7f4952e3ae8d88b22aed957ef44a38df68b42abfd934f322f81710ef2d55f4"} Jun 10 12:52:37.894722 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.894360 2582 scope.go:117] "RemoveContainer" containerID="3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6" Jun 10 12:52:37.904541 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.904520 2582 scope.go:117] "RemoveContainer" containerID="3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6" Jun 10 12:52:37.904847 ip-10-0-138-99 kubenswrapper[2582]: E0610 12:52:37.904829 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6\": container with ID starting with 3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6 not found: ID does not exist" containerID="3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6" Jun 10 12:52:37.904894 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.904857 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6"} err="failed to get container status \"3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6\": rpc error: code = NotFound desc = could not find container \"3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6\": container with ID starting with 3343a60fa92e2cbde61f543df1841f6834048b0bdf78d47d778a648b6be63ba6 not found: ID does not exist" Jun 10 12:52:37.918649 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.918619 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7756944cf8-cvd5q"] Jun 10 12:52:37.923270 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:37.923231 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7756944cf8-cvd5q"] Jun 10 12:52:38.248283 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:52:38.248186 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec616f84-5301-4afa-a2c9-df4154367be8" path="/var/lib/kubelet/pods/ec616f84-5301-4afa-a2c9-df4154367be8/volumes" Jun 10 12:56:50.327606 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:56:50.327572 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:56:50.331906 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:56:50.331881 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tfsmb_ee00a67b-e3d6-4c47-81dc-d5a523e04dc7/ovn-acl-logging/0.log" Jun 10 12:57:45.690972 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:57:45.690938 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:53.623970 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:57:53.623923 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:07.158938 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:07.158854 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:12.210829 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:12.210794 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:33.094919 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:33.094884 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:53.185944 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:53.185910 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-f676969f4-7fsr8_4ad6ffbe-3a4c-455c-8a24-472c123a04bb/nginx/0.log" Jun 10 12:59:53.452558 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:53.452470 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-vzc9b_1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e/manager/0.log" Jun 10 12:59:53.585520 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:53.585493 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-w5q7d_ec14a5bb-eca7-4851-9df2-61f5ab7bc066/manager/0.log" Jun 10 12:59:53.852391 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:53.852366 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-cpfzp_aa0cfac9-cacd-4e3d-954a-757de8e42280/server/0.log" Jun 10 12:59:53.978230 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:53.978198 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-zwbrs_4a81b239-4f98-42d6-8de2-c1e069633bc5/manager/0.log" Jun 10 12:59:54.220428 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:54.220344 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c7bd9b77-cqgr7_d281b21d-ebd1-4296-9694-f4fcb17d9805/copy-manifests/0.log" Jun 10 12:59:54.240456 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:54.240422 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c7bd9b77-cqgr7_d281b21d-ebd1-4296-9694-f4fcb17d9805/manager/0.log" Jun 10 12:59:55.130169 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.130135 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl_a1c12211-3534-4788-b5bd-39a0be5843fd/util/0.log" Jun 10 12:59:55.137761 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.137734 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl_a1c12211-3534-4788-b5bd-39a0be5843fd/pull/0.log" Jun 10 12:59:55.144691 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.144664 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl_a1c12211-3534-4788-b5bd-39a0be5843fd/extract/0.log" Jun 10 12:59:55.257118 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.257075 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l_03d63aba-abc2-4dc1-933c-71f1d0cb29f7/util/0.log" Jun 10 12:59:55.265091 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.265068 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l_03d63aba-abc2-4dc1-933c-71f1d0cb29f7/pull/0.log" Jun 10 12:59:55.275215 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.275195 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l_03d63aba-abc2-4dc1-933c-71f1d0cb29f7/extract/0.log" Jun 10 12:59:55.391554 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.391478 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5_23d17881-2f4f-4655-83df-8979581837e6/util/0.log" Jun 10 12:59:55.398895 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.398875 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5_23d17881-2f4f-4655-83df-8979581837e6/pull/0.log" Jun 10 12:59:55.406230 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.406204 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5_23d17881-2f4f-4655-83df-8979581837e6/extract/0.log" Jun 10 12:59:55.519616 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.519588 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr_527361d7-e4c6-4f05-90b1-560e11228c77/util/0.log" Jun 10 12:59:55.528599 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.528557 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr_527361d7-e4c6-4f05-90b1-560e11228c77/pull/0.log" Jun 10 12:59:55.535389 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:55.535366 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr_527361d7-e4c6-4f05-90b1-560e11228c77/extract/0.log" Jun 10 12:59:56.330996 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:56.330926 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-srx49_d904d244-e090-43bc-be52-99409a02790f/limitador/0.log" Jun 10 12:59:56.450526 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:56.450492 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vnbs8_30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4/manager/0.log" Jun 10 12:59:56.926879 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:56.926848 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-pbvl9_e2def4ea-79e3-4a9f-884c-a3fd1d95554a/discovery/0.log" Jun 10 12:59:57.154770 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:57.154731 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7567468478-wz9pp_70700fe6-5cd4-4a4b-b646-ccc3c4eee484/kube-auth-proxy/0.log" Jun 10 12:59:57.493364 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:57.493334 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-ss2qm_13395be0-f360-4121-b1a4-293a6aec2163/payload-processing/0.log" Jun 10 12:59:57.605843 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:57.605814 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6776685957-2pxv2_5c1b093b-3292-460d-909a-145a7acfc796/router/0.log" Jun 10 12:59:58.072203 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:58.072170 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-df8g2_29b02545-4a49-48fd-ba87-d09b27252bde/storage-initializer/0.log" Jun 10 12:59:58.080580 ip-10-0-138-99 kubenswrapper[2582]: I0610 12:59:58.080556 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-df8g2_29b02545-4a49-48fd-ba87-d09b27252bde/main/0.log" Jun 10 13:00:00.158547 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.158506 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d"] Jun 10 13:00:00.158938 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.158911 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec616f84-5301-4afa-a2c9-df4154367be8" containerName="console" Jun 10 13:00:00.158938 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.158922 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec616f84-5301-4afa-a2c9-df4154367be8" containerName="console" Jun 10 13:00:00.159016 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.159005 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec616f84-5301-4afa-a2c9-df4154367be8" containerName="console" Jun 10 13:00:00.162238 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.162220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d" Jun 10 13:00:00.165440 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.165422 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jun 10 13:00:00.166871 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.166848 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jun 10 13:00:00.166988 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.166850 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-dockercfg-b9sbg\"" Jun 10 13:00:00.184267 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.184226 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d"] Jun 10 13:00:00.255564 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.255535 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmfgj\" (UniqueName: \"kubernetes.io/projected/1fc6a69f-e74e-45c9-bc6d-aea65d951f18-kube-api-access-gmfgj\") pod \"maas-api-key-cleanup-29684940-ds26d\" (UID: \"1fc6a69f-e74e-45c9-bc6d-aea65d951f18\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d" Jun 10 13:00:00.356833 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.356797 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmfgj\" (UniqueName: \"kubernetes.io/projected/1fc6a69f-e74e-45c9-bc6d-aea65d951f18-kube-api-access-gmfgj\") pod \"maas-api-key-cleanup-29684940-ds26d\" (UID: \"1fc6a69f-e74e-45c9-bc6d-aea65d951f18\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d" Jun 10 13:00:00.366512 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.366481 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmfgj\" (UniqueName: \"kubernetes.io/projected/1fc6a69f-e74e-45c9-bc6d-aea65d951f18-kube-api-access-gmfgj\") pod \"maas-api-key-cleanup-29684940-ds26d\" (UID: \"1fc6a69f-e74e-45c9-bc6d-aea65d951f18\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d" Jun 10 13:00:00.472802 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.472708 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d" Jun 10 13:00:00.614207 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.614179 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d"] Jun 10 13:00:00.615874 ip-10-0-138-99 kubenswrapper[2582]: W0610 13:00:00.615846 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc6a69f_e74e_45c9_bc6d_aea65d951f18.slice/crio-1e734cfc28aa79470c2c0f546065c2b01084237fddbde678e4114ff949f99fc2 WatchSource:0}: Error finding container 1e734cfc28aa79470c2c0f546065c2b01084237fddbde678e4114ff949f99fc2: Status 404 returned error can't find the container with id 1e734cfc28aa79470c2c0f546065c2b01084237fddbde678e4114ff949f99fc2 Jun 10 13:00:00.618175 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:00.618160 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 13:00:01.517789 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:01.517751 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d" event={"ID":"1fc6a69f-e74e-45c9-bc6d-aea65d951f18","Type":"ContainerStarted","Data":"1e734cfc28aa79470c2c0f546065c2b01084237fddbde678e4114ff949f99fc2"} Jun 10 13:00:02.683379 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:02.683342 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:02.791518 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:02.791474 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:05.680985 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:05.680955 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fww7p_63acd877-6555-4895-8e27-9b7540f95de8/global-pull-secret-syncer/0.log" Jun 10 13:00:05.785464 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:05.785431 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-99zc2_9303f166-88cd-4695-8a92-ead714e32ca1/konnectivity-agent/0.log" Jun 10 13:00:05.914595 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:05.914566 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-99.ec2.internal_89d633427a813ac026468ec864f4e82d/haproxy/0.log" Jun 10 13:00:10.107799 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.107772 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-fxc66_5f4af62a-89c8-4e44-82b5-204cc9a78a8a/sail-operator/0.log" Jun 10 13:00:10.656611 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.656571 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl_a1c12211-3534-4788-b5bd-39a0be5843fd/extract/0.log" Jun 10 13:00:10.689246 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.689221 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl_a1c12211-3534-4788-b5bd-39a0be5843fd/util/0.log" Jun 10 13:00:10.721768 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.721729 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wlsjl_a1c12211-3534-4788-b5bd-39a0be5843fd/pull/0.log" Jun 10 13:00:10.753801 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.753771 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l_03d63aba-abc2-4dc1-933c-71f1d0cb29f7/extract/0.log" Jun 10 13:00:10.784525 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.784493 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l_03d63aba-abc2-4dc1-933c-71f1d0cb29f7/util/0.log" Jun 10 13:00:10.807875 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.807848 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09c78l_03d63aba-abc2-4dc1-933c-71f1d0cb29f7/pull/0.log" Jun 10 13:00:10.866178 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.866150 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5_23d17881-2f4f-4655-83df-8979581837e6/extract/0.log" Jun 10 13:00:10.893366 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.893339 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5_23d17881-2f4f-4655-83df-8979581837e6/util/0.log" Jun 10 13:00:10.922302 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.922233 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lwbn5_23d17881-2f4f-4655-83df-8979581837e6/pull/0.log" Jun 10 13:00:10.952913 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.952879 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr_527361d7-e4c6-4f05-90b1-560e11228c77/extract/0.log" Jun 10 13:00:10.976776 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:10.976743 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr_527361d7-e4c6-4f05-90b1-560e11228c77/util/0.log" Jun 10 13:00:11.007439 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:11.007413 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wv8wr_527361d7-e4c6-4f05-90b1-560e11228c77/pull/0.log" Jun 10 13:00:11.254007 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:11.253910 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-srx49_d904d244-e090-43bc-be52-99409a02790f/limitador/0.log" Jun 10 13:00:11.323714 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:11.323679 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vnbs8_30c5fe3c-9ddd-4b40-93a0-09b3baa64bc4/manager/0.log" Jun 10 13:00:13.126958 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.126913 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3df88ada-e928-4047-ad3a-2a01f762add1/alertmanager/0.log" Jun 10 13:00:13.154055 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.153972 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3df88ada-e928-4047-ad3a-2a01f762add1/config-reloader/0.log" Jun 10 13:00:13.186567 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.186538 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3df88ada-e928-4047-ad3a-2a01f762add1/kube-rbac-proxy-web/0.log" Jun 10 13:00:13.215255 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.215223 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3df88ada-e928-4047-ad3a-2a01f762add1/kube-rbac-proxy/0.log" Jun 10 13:00:13.243625 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.243596 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3df88ada-e928-4047-ad3a-2a01f762add1/kube-rbac-proxy-metric/0.log" Jun 10 13:00:13.282418 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.282382 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3df88ada-e928-4047-ad3a-2a01f762add1/prom-label-proxy/0.log" Jun 10 13:00:13.327110 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.327065 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3df88ada-e928-4047-ad3a-2a01f762add1/init-config-reloader/0.log" Jun 10 13:00:13.414059 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.413978 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-pd7jx_2d630daa-7080-4ae1-b427-0a901405386a/kube-state-metrics/0.log" Jun 10 13:00:13.438931 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.438885 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-pd7jx_2d630daa-7080-4ae1-b427-0a901405386a/kube-rbac-proxy-main/0.log" Jun 10 13:00:13.465217 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.465184 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-pd7jx_2d630daa-7080-4ae1-b427-0a901405386a/kube-rbac-proxy-self/0.log" Jun 10 13:00:13.508379 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.508353 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-867d7ffb85-s2wvx_dc283677-d4ee-4fab-a89f-e09a25a8cef6/metrics-server/0.log" Jun 10 13:00:13.544815 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.544783 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-p6mrp_9f6621df-fea0-4d8b-95a9-b48be65a38cd/monitoring-plugin/0.log" Jun 10 13:00:13.579944 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.579914 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjz24_ccb06a3a-3837-4662-bbfc-a2b80b648493/node-exporter/0.log" Jun 10 13:00:13.602323 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.602293 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjz24_ccb06a3a-3837-4662-bbfc-a2b80b648493/kube-rbac-proxy/0.log" Jun 10 13:00:13.630048 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:13.630026 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjz24_ccb06a3a-3837-4662-bbfc-a2b80b648493/init-textfile/0.log" Jun 10 13:00:15.624276 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:15.624240 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-mx9nj_3e8c9935-3c74-4f7f-a0d5-81b1abdfa7c0/networking-console-plugin/0.log" Jun 10 13:00:15.945190 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:15.945118 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv"] Jun 10 13:00:15.948721 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:15.948701 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:15.952392 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:15.952373 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-42cd4\"/\"kube-root-ca.crt\"" Jun 10 13:00:15.952676 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:15.952644 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-42cd4\"/\"openshift-service-ca.crt\"" Jun 10 13:00:15.953796 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:15.953779 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-42cd4\"/\"default-dockercfg-4pwrt\"" Jun 10 13:00:15.963792 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:15.963769 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv"] Jun 10 13:00:16.099677 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.099633 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fwhl\" (UniqueName: \"kubernetes.io/projected/2e1aa545-2699-4585-bc58-2cf8186927d5-kube-api-access-9fwhl\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.099860 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.099687 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-proc\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.099860 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.099752 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-sys\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.099978 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.099853 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-lib-modules\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.099978 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.099898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-podres\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.200730 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.200650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-lib-modules\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.200730 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.200692 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-podres\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.200933 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.200748 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fwhl\" (UniqueName: \"kubernetes.io/projected/2e1aa545-2699-4585-bc58-2cf8186927d5-kube-api-access-9fwhl\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.200933 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.200766 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-proc\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.200933 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.200796 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-sys\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.200933 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.200846 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-lib-modules\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.200933 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.200873 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-sys\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.200933 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.200851 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-podres\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.200933 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.200884 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e1aa545-2699-4585-bc58-2cf8186927d5-proc\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.214200 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.214174 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fwhl\" (UniqueName: \"kubernetes.io/projected/2e1aa545-2699-4585-bc58-2cf8186927d5-kube-api-access-9fwhl\") pod \"perf-node-gather-daemonset-lmzsv\" (UID: \"2e1aa545-2699-4585-bc58-2cf8186927d5\") " pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.262324 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.262295 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:16.956657 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:16.956629 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6996bff5fb-d4j5m_723aa781-0e4d-413d-8f8f-65023f6c378b/console/0.log" Jun 10 13:00:17.032858 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:17.032830 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-f8995_066f8f06-a2e4-4703-9e42-ce8067800a2c/download-server/0.log" Jun 10 13:00:17.040214 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:17.040192 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv"] Jun 10 13:00:17.041412 ip-10-0-138-99 kubenswrapper[2582]: W0610 13:00:17.041383 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2e1aa545_2699_4585_bc58_2cf8186927d5.slice/crio-b415d8a9cd1afc96e129dd8c3abdd1e5e3401f0fac8f9cc1a19a84309b830493 WatchSource:0}: Error finding container b415d8a9cd1afc96e129dd8c3abdd1e5e3401f0fac8f9cc1a19a84309b830493: Status 404 returned error can't find the container with id b415d8a9cd1afc96e129dd8c3abdd1e5e3401f0fac8f9cc1a19a84309b830493 Jun 10 13:00:17.618212 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:17.618169 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d" event={"ID":"1fc6a69f-e74e-45c9-bc6d-aea65d951f18","Type":"ContainerStarted","Data":"e46bb405a01d6e2781cf55f82769c12f2b38e890eabd35633aee7a8eed36e471"} Jun 10 13:00:17.619723 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:17.619695 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" event={"ID":"2e1aa545-2699-4585-bc58-2cf8186927d5","Type":"ContainerStarted","Data":"8fcf229d5cb17d769f419831e663a5b74a6bfc0c2fb73ea07c3de70b5579b835"} Jun 10 13:00:17.619848 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:17.619742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" event={"ID":"2e1aa545-2699-4585-bc58-2cf8186927d5","Type":"ContainerStarted","Data":"b415d8a9cd1afc96e129dd8c3abdd1e5e3401f0fac8f9cc1a19a84309b830493"} Jun 10 13:00:17.619848 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:17.619765 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:17.662884 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:17.662831 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29684940-ds26d" podStartSLOduration=1.332915519 podStartE2EDuration="17.662812901s" totalImagesPullingTime="16.329897382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:00:17.662774747 +0000 UTC m=+1407.873620737" watchObservedRunningTime="2026-06-10 13:00:17.662812901 +0000 UTC m=+1407.873658892" Jun 10 13:00:17.712390 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:17.712337 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" podStartSLOduration=2.7123199590000002 podStartE2EDuration="2.712319959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 13:00:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:00:17.71097277 +0000 UTC m=+1407.921818758" watchObservedRunningTime="2026-06-10 13:00:17.712319959 +0000 UTC m=+1407.923165945" Jun 10 13:00:17.928764 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:17.928675 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-48nxw_cdfee373-6546-49da-87ce-2c92603387ca/volume-data-source-validator/0.log" Jun 10 13:00:19.192058 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:19.192023 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmlx9_306d7497-fa10-42e3-8c0a-7fb50bf91206/dns/0.log" Jun 10 13:00:19.216983 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:19.216954 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmlx9_306d7497-fa10-42e3-8c0a-7fb50bf91206/kube-rbac-proxy/0.log" Jun 10 13:00:19.244716 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:19.244685 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5m4pd_666e8e25-9dde-4db2-a3e2-3e75b5e24059/dns-node-resolver/0.log" Jun 10 13:00:19.971280 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:19.971247 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5d59c77bf7-bcjl2_6d476b41-3a3c-42e3-aa56-db0425aa0093/registry/0.log" Jun 10 13:00:20.025590 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:20.025564 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gqpm5_af9f4f53-a664-4c7d-a457-9e0779ad3c16/node-ca/0.log" Jun 10 13:00:21.162774 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:21.162739 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-pbvl9_e2def4ea-79e3-4a9f-884c-a3fd1d95554a/discovery/0.log" Jun 10 13:00:21.219538 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:21.219510 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7567468478-wz9pp_70700fe6-5cd4-4a4b-b646-ccc3c4eee484/kube-auth-proxy/0.log" Jun 10 13:00:21.324034 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:21.324006 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-ss2qm_13395be0-f360-4121-b1a4-293a6aec2163/payload-processing/0.log" Jun 10 13:00:21.363446 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:21.363410 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6776685957-2pxv2_5c1b093b-3292-460d-909a-145a7acfc796/router/0.log" Jun 10 13:00:22.085535 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:22.085499 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f5d7t_95d21008-75a7-4b76-8a26-ede242d95fd4/serve-healthcheck-canary/0.log" Jun 10 13:00:22.972914 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:22.972883 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dzbg8_052eabd0-4b09-475f-bdb9-b01d4b8fe0ea/kube-rbac-proxy/0.log" Jun 10 13:00:22.999395 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:22.999338 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dzbg8_052eabd0-4b09-475f-bdb9-b01d4b8fe0ea/exporter/0.log" Jun 10 13:00:23.025940 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:23.025913 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dzbg8_052eabd0-4b09-475f-bdb9-b01d4b8fe0ea/extractor/0.log" Jun 10 13:00:23.635262 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:23.635228 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-42cd4/perf-node-gather-daemonset-lmzsv" Jun 10 13:00:25.697136 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:25.697062 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-f676969f4-7fsr8_4ad6ffbe-3a4c-455c-8a24-472c123a04bb/nginx/0.log" Jun 10 13:00:25.782691 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:25.782656 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-vzc9b_1673ed4a-a2cf-4f5c-9aad-5e17a1c0393e/manager/0.log" Jun 10 13:00:25.843764 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:25.843726 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-w5q7d_ec14a5bb-eca7-4851-9df2-61f5ab7bc066/manager/0.log" Jun 10 13:00:25.929258 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:25.929229 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-cpfzp_aa0cfac9-cacd-4e3d-954a-757de8e42280/server/0.log" Jun 10 13:00:26.032309 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:26.032276 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-zwbrs_4a81b239-4f98-42d6-8de2-c1e069633bc5/manager/0.log" Jun 10 13:00:26.248659 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:26.248634 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c7bd9b77-cqgr7_d281b21d-ebd1-4296-9694-f4fcb17d9805/manager/0.log" Jun 10 13:00:26.302786 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:26.302761 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c7bd9b77-cqgr7_d281b21d-ebd1-4296-9694-f4fcb17d9805/copy-manifests/0.log" Jun 10 13:00:28.229547 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:28.229445 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-g2b2c_71ef3587-45e9-4489-8cf3-22eff70a970e/openshift-lws-operator/0.log" Jun 10 13:00:30.640992 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:30.640900 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-pbvl9_e2def4ea-79e3-4a9f-884c-a3fd1d95554a/discovery/0.log" Jun 10 13:00:30.676091 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:30.676058 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7567468478-wz9pp_70700fe6-5cd4-4a4b-b646-ccc3c4eee484/kube-auth-proxy/0.log" Jun 10 13:00:30.741904 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:30.741871 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-ss2qm_13395be0-f360-4121-b1a4-293a6aec2163/payload-processing/0.log" Jun 10 13:00:30.768509 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:30.768477 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6776685957-2pxv2_5c1b093b-3292-460d-909a-145a7acfc796/router/0.log" Jun 10 13:00:31.864243 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:31.864209 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-pbvl9_e2def4ea-79e3-4a9f-884c-a3fd1d95554a/discovery/0.log" Jun 10 13:00:31.895838 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:31.895801 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7567468478-wz9pp_70700fe6-5cd4-4a4b-b646-ccc3c4eee484/kube-auth-proxy/0.log" Jun 10 13:00:31.951156 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:31.951126 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-ss2qm_13395be0-f360-4121-b1a4-293a6aec2163/payload-processing/0.log" Jun 10 13:00:31.970610 ip-10-0-138-99 kubenswrapper[2582]: I0610 13:00:31.970583 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6776685957-2pxv2_5c1b093b-3292-460d-909a-145a7acfc796/router/0.log"