Jun 08 22:47:58.072791 ip-10-0-138-248 systemd[1]: Starting Kubernetes Kubelet... Jun 08 22:47:58.461903 ip-10-0-138-248 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 22:47:58.461903 ip-10-0-138-248 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 08 22:47:58.461903 ip-10-0-138-248 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 22:47:58.461903 ip-10-0-138-248 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 08 22:47:58.461903 ip-10-0-138-248 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 22:47:58.462632 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.461957 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 08 22:47:58.466223 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466206 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.466223 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466222 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466226 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466229 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466232 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466235 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466239 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466242 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466245 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466252 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466256 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466258 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466261 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466264 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466267 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466270 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466272 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466275 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466278 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466280 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466283 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.466287 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466286 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466288 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466291 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466294 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466297 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466300 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466303 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466305 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466308 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466310 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466313 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466316 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466318 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466321 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466323 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466326 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466329 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466332 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466334 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466337 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.466778 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466339 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466342 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466344 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466348 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466352 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466355 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466358 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466361 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466364 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466366 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466369 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466372 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466375 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466378 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466382 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466385 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466399 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466402 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466405 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466408 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.467252 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466411 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466414 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466417 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466419 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466422 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466425 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466428 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466433 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466436 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466439 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466442 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466445 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466447 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466450 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466452 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466455 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466458 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466460 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466463 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.467757 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466466 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466469 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466471 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466474 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466477 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466479 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466483 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466486 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466489 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466492 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466494 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466499 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466503 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466506 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466508 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466512 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466514 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466517 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.466520 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467246 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.468221 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467252 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467255 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467258 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467261 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467267 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467279 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467282 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467285 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467288 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467291 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467295 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467298 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467300 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467303 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467306 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467308 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467311 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467314 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467316 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467319 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.468737 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467321 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467324 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467327 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467330 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467332 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467335 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467337 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467340 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467343 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467345 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467348 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467351 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467353 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467356 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467358 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467362 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467366 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467370 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467373 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467376 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.469233 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467378 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467382 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467385 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467403 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467407 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467411 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467415 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467418 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467420 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467423 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467426 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467429 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467432 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467435 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467437 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467440 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467442 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467445 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467447 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.469723 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467450 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467453 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467455 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467458 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467461 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467463 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467466 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467469 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467472 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467474 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467481 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467484 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467486 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467489 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467493 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467495 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467498 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467500 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467503 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467505 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.470187 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467508 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467511 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467513 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467516 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467518 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467521 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467523 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467527 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467531 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467533 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467536 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467538 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467543 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467546 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467548 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467551 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467553 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467556 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467558 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467988 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.470688 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467994 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.467997 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468002 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468006 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468009 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468012 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468014 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468017 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468019 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468022 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468025 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468027 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468030 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468033 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468035 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468038 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468040 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468043 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468045 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468048 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.471194 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468051 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468053 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468056 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468059 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468062 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468065 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468067 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468070 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468073 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468075 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468078 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468080 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468083 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468085 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468089 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468092 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468095 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468098 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468101 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468103 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.471724 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468106 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468109 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468111 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468114 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468116 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468119 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468121 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468124 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468126 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468129 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468133 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468137 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468140 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468143 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468146 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468149 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468153 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468156 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468159 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.472203 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468162 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468165 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468168 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468170 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468173 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468176 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468178 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468182 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468185 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468187 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468190 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468193 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468196 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468198 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468202 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468204 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468207 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468210 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468213 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468216 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.472745 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468219 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468222 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468225 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468229 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468233 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468237 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468240 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468242 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468245 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468249 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468251 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468254 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468257 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468259 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468262 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468264 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468267 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468269 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468271 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468343 2567 flags.go:64] FLAG: --address="0.0.0.0" Jun 08 22:47:58.473231 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468351 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468358 2567 flags.go:64] FLAG: --anonymous-auth="true" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468371 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468376 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468379 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468383 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468388 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468404 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468407 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468411 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468414 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468418 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468421 2567 flags.go:64] FLAG: --cgroup-root="" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468424 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468427 2567 flags.go:64] FLAG: --client-ca-file="" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468429 2567 flags.go:64] FLAG: --cloud-provider="external" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468432 2567 flags.go:64] FLAG: --cluster-dns="[]" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468436 2567 flags.go:64] FLAG: --cluster-domain="" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468439 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468442 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468445 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468451 2567 flags.go:64] FLAG: --container-log-max-files="5" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468455 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 08 22:47:58.473768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468458 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468461 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468465 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468468 2567 flags.go:64] FLAG: --contention-profiling="false" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468471 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468474 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468478 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468481 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468486 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468489 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468492 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468495 2567 flags.go:64] FLAG: --enable-load-reader="false" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468498 2567 flags.go:64] FLAG: --enable-server="true" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468502 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468507 2567 flags.go:64] FLAG: --event-burst="100" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468510 2567 flags.go:64] FLAG: --event-qps="50" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468512 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468520 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468523 2567 flags.go:64] FLAG: --eviction-hard="" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468527 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468530 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468533 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468536 2567 flags.go:64] FLAG: --eviction-soft="" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468539 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468542 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 08 22:47:58.474330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468546 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468548 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468551 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468554 2567 flags.go:64] FLAG: --fail-swap-on="true" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468557 2567 flags.go:64] FLAG: --feature-gates="" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468562 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468565 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468568 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468571 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468575 2567 flags.go:64] FLAG: --healthz-port="10248" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468578 2567 flags.go:64] FLAG: --help="false" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468581 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468584 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468587 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468590 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468594 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468598 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468602 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468604 2567 flags.go:64] FLAG: --image-service-endpoint="" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468607 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468610 2567 flags.go:64] FLAG: --kube-api-burst="100" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468614 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468617 2567 flags.go:64] FLAG: --kube-api-qps="50" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468620 2567 flags.go:64] FLAG: --kube-reserved="" Jun 08 22:47:58.474992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468622 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468625 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468628 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468631 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468634 2567 flags.go:64] FLAG: --lock-file="" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468637 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468640 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468643 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468648 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468651 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468654 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468657 2567 flags.go:64] FLAG: --logging-format="text" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468660 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468664 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468668 2567 flags.go:64] FLAG: --manifest-url="" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468670 2567 flags.go:64] FLAG: --manifest-url-header="" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468675 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468678 2567 flags.go:64] FLAG: --max-open-files="1000000" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468682 2567 flags.go:64] FLAG: --max-pods="110" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468685 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468688 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468691 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468694 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468698 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468701 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 08 22:47:58.475690 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468704 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468711 2567 flags.go:64] FLAG: --node-status-max-images="50" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468714 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468717 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468721 2567 flags.go:64] FLAG: --pod-cidr="" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468724 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468730 2567 flags.go:64] FLAG: --pod-manifest-path="" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468733 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468736 2567 flags.go:64] FLAG: --pods-per-core="0" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468739 2567 flags.go:64] FLAG: --port="10250" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468742 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468745 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-025f792206e97b75c" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468749 2567 flags.go:64] FLAG: --qos-reserved="" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468752 2567 flags.go:64] FLAG: --read-only-port="10255" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468755 2567 flags.go:64] FLAG: --register-node="true" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468758 2567 flags.go:64] FLAG: --register-with-taints="" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468761 2567 flags.go:64] FLAG: --registry-burst="10" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468764 2567 flags.go:64] FLAG: --registry-qps="5" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468767 2567 flags.go:64] FLAG: --reserved-cpus="" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468769 2567 flags.go:64] FLAG: --reserved-memory="" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468774 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468777 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468780 2567 flags.go:64] FLAG: --rotate-certificates="false" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468783 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468786 2567 flags.go:64] FLAG: --runonce="false" Jun 08 22:47:58.476301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468789 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468791 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468794 2567 flags.go:64] FLAG: --seccomp-default="false" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468797 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468800 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468803 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468807 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468810 2567 flags.go:64] FLAG: --storage-driver-password="root" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468813 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468816 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468818 2567 flags.go:64] FLAG: --storage-driver-user="root" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468821 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468824 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468833 2567 flags.go:64] FLAG: --system-cgroups="" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468836 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468841 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468844 2567 flags.go:64] FLAG: --tls-cert-file="" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468847 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468851 2567 flags.go:64] FLAG: --tls-min-version="" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468854 2567 flags.go:64] FLAG: --tls-private-key-file="" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468857 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468860 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468863 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468866 2567 flags.go:64] FLAG: --v="2" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468870 2567 flags.go:64] FLAG: --version="false" Jun 08 22:47:58.476917 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468874 2567 flags.go:64] FLAG: --vmodule="" Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468878 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.468881 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468995 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.468998 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469001 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469004 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469007 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469010 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469013 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469016 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469018 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469021 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469024 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469027 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469029 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469032 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469036 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469040 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469043 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.477534 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469046 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469049 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469052 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469055 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469058 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469061 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469064 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469067 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469070 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469072 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469075 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469078 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469080 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469083 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469085 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469088 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469090 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469093 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469096 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469098 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.478020 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469101 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469103 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469106 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469108 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469111 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469114 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469116 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469120 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469123 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469126 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469129 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469131 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469134 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469136 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469139 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469142 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469145 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469147 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469150 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469152 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.478525 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469155 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469157 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469160 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469162 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469164 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469167 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469170 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469172 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469175 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469177 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469179 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469182 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469185 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469187 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469190 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469192 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469195 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469198 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469200 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.479073 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469202 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469205 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469208 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469210 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469213 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469216 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469218 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469220 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469224 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469226 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469230 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469233 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469236 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469239 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469243 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469246 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469249 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469252 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469255 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.479649 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469258 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469261 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469265 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.469267 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.469728 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.476076 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.476091 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.476111 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.476117 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476172 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476176 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476180 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476182 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476185 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476188 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.480105 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476190 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476193 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476196 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476199 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476201 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476204 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476207 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476210 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476212 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476215 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476218 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476221 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476223 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476226 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476228 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476231 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476233 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476236 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476239 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476241 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.480523 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476244 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476247 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476249 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476254 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476256 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476259 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476262 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476264 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476267 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476269 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476272 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476274 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476277 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476280 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476282 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476286 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476290 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476293 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476295 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476298 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.481037 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476301 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476304 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476306 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476310 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476312 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476315 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476318 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476320 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476325 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476328 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476331 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476334 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476336 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476339 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476341 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476345 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476348 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476351 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476353 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476356 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.481529 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476358 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476362 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476365 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476369 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476371 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476374 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476377 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476379 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476382 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476385 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476402 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476407 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476410 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476413 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476416 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476418 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476421 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476423 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476426 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.482028 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476428 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476431 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476433 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476436 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476438 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476441 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476444 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476446 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476449 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476452 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476456 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476459 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476461 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476464 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.476469 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 22:47:58.482499 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476592 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476597 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476601 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476604 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476606 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476610 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476613 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476615 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476618 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476622 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476624 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476627 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476630 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476632 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476635 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476637 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476640 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476642 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476645 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476647 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.482860 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476650 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476652 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476655 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476657 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476660 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476662 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476666 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476668 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476671 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476674 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476676 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476678 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476681 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476683 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476686 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476688 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476691 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476693 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476696 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476698 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.483338 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476700 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476703 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476705 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476708 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476710 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476713 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476715 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476718 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476720 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476723 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476725 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476728 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476730 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476733 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476735 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476738 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476741 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476743 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476746 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476748 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.483837 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476751 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476754 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476756 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476758 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476761 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476763 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476766 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476768 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476772 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476776 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476779 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476782 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476785 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476788 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476791 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476793 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476797 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476800 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.484326 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476803 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476807 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476810 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476813 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476815 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476818 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476821 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476824 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476827 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476830 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476832 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476835 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476838 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476840 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476843 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476846 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476849 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476852 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476854 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.484887 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476857 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.485788 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:58.476859 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.485788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.476864 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 22:47:58.485788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.476971 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jun 08 22:47:58.485788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.480201 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 08 22:47:58.485788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.481489 2567 server.go:1019] "Starting client certificate rotation" Jun 08 22:47:58.485788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.481588 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 22:47:58.485788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.481656 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 22:47:58.486961 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.486938 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 22:47:58.488215 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.488199 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 22:47:58.504068 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.504050 2567 log.go:25] "Validated CRI v1 runtime API" Jun 08 22:47:58.506898 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.506871 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 22:47:58.507201 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.507177 2567 log.go:25] "Validated CRI v1 image API" Jun 08 22:47:58.508666 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.508651 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 08 22:47:58.511222 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.511204 2567 fs.go:135] Filesystem UUIDs: map[1c1d1e8d-690d-43f8-92e0-4d1dd934e707:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ef9c5b84-e940-4d97-8af8-7c8c52403f72:/dev/nvme0n1p3] Jun 08 22:47:58.511295 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.511223 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 08 22:47:58.517044 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.516942 2567 manager.go:216] Machine: {Timestamp:2026-06-08 22:47:58.515687718 +0000 UTC m=+0.324299581 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099908 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26481bd6e821052edc258b2177a5a2 SystemUUID:ec26481b-d6e8-2105-2edc-258b2177a5a2 BootID:fc367335-6791-4f44-afd5-4340f4b82a29 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:98:a2:83:32:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:98:a2:83:32:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:62:3d:e0:07:30 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 08 22:47:58.517044 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.517042 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 08 22:47:58.517151 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.517123 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 08 22:47:58.520202 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.520173 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 08 22:47:58.520340 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.520204 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-248.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 08 22:47:58.520926 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.520916 2567 topology_manager.go:138] "Creating topology manager with none policy" Jun 08 22:47:58.520956 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.520928 2567 container_manager_linux.go:309] "Creating device plugin manager" Jun 08 22:47:58.520956 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.520936 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 22:47:58.521730 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.521718 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 22:47:58.522512 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.522503 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 08 22:47:58.525018 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.525007 2567 state_mem.go:36] "Initialized new in-memory state store" Jun 08 22:47:58.525130 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.525121 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 08 22:47:58.525167 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.525153 2567 kubelet.go:486] "Attempting to sync node with API server" Jun 08 22:47:58.525195 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.525167 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 08 22:47:58.525195 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.525178 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 08 22:47:58.525195 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.525188 2567 kubelet.go:398] "Adding apiserver pod source" Jun 08 22:47:58.525274 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.525196 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 08 22:47:58.525595 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.525582 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 22:47:58.525644 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.525606 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 22:47:58.527956 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.527936 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 08 22:47:58.530404 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.530369 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 08 22:47:58.530490 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.530419 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 08 22:47:58.531489 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531471 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 08 22:47:58.531594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531495 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 08 22:47:58.531594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531505 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 08 22:47:58.531594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531514 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 08 22:47:58.531594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531529 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 08 22:47:58.531594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531546 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 08 22:47:58.531594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531560 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 08 22:47:58.531594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531569 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 08 22:47:58.531594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531581 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 08 22:47:58.531594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531590 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 08 22:47:58.531897 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531637 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 08 22:47:58.531897 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.531648 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 08 22:47:58.532460 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.532442 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 08 22:47:58.532539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.532472 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 08 22:47:58.533299 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.533273 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 08 22:47:58.533490 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.533472 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 08 22:47:58.536021 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.536003 2567 server.go:1268] "Started kubelet" Jun 08 22:47:58.536103 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.536073 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 08 22:47:58.536163 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.536080 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 08 22:47:58.536163 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.536149 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 08 22:47:58.536952 ip-10-0-138-248 systemd[1]: Started Kubernetes Kubelet. Jun 08 22:47:58.537343 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.537286 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-248.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 08 22:47:58.538793 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.538774 2567 server.go:310] "Adding debug handlers to kubelet server" Jun 08 22:47:58.540587 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.540569 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 08 22:47:58.542622 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.542577 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 08 22:47:58.542752 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.541815 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-248.ec2.internal.18b73caa02b6deba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-248.ec2.internal,UID:ip-10-0-138-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-248.ec2.internal,},FirstTimestamp:2026-06-08 22:47:58.535966394 +0000 UTC m=+0.344578239,LastTimestamp:2026-06-08 22:47:58.535966394 +0000 UTC m=+0.344578239,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-248.ec2.internal,}" Jun 08 22:47:58.544874 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.544855 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 08 22:47:58.544874 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.544865 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 08 22:47:58.545402 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.545181 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:58.545483 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.545448 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 08 22:47:58.545622 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.545604 2567 factory.go:55] Registering systemd factory Jun 08 22:47:58.545706 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.545626 2567 factory.go:223] Registration of the systemd container factory successfully Jun 08 22:47:58.545706 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.545597 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 08 22:47:58.545706 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.545681 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 08 22:47:58.545860 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.545812 2567 reconstruct.go:97] "Volume reconstruction finished" Jun 08 22:47:58.545860 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.545821 2567 reconciler.go:29] "Reconciler: start to sync state" Jun 08 22:47:58.545966 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.545951 2567 factory.go:153] Registering CRI-O factory Jun 08 22:47:58.546009 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.545977 2567 factory.go:223] Registration of the crio container factory successfully Jun 08 22:47:58.546062 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.546031 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 08 22:47:58.546062 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.546052 2567 factory.go:103] Registering Raw factory Jun 08 22:47:58.546154 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.546068 2567 manager.go:1196] Started watching for new ooms in manager Jun 08 22:47:58.546539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.546525 2567 manager.go:349] Starting recovery of all containers Jun 08 22:47:58.546746 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.546721 2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 08 22:47:58.546855 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.546835 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 08 22:47:58.556888 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.556773 2567 manager.go:354] Recovery completed Jun 08 22:47:58.560767 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.560746 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-blv6r" Jun 08 22:47:58.563260 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.563248 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.565186 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.565171 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.565260 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.565205 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.565260 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.565221 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.565765 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.565747 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 08 22:47:58.565765 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.565759 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 08 22:47:58.565876 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.565776 2567 state_mem.go:36] "Initialized new in-memory state store" Jun 08 22:47:58.565876 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.565797 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.566586 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.566479 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.566586 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.566499 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.566586 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.566510 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.566904 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.566893 2567 policy_none.go:49] "None policy: Start" Jun 08 22:47:58.566941 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.566909 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 08 22:47:58.566941 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.566920 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 08 22:47:58.567340 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.567278 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-248.ec2.internal.18b73caa0474bd59 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-248.ec2.internal,UID:ip-10-0-138-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-248.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-248.ec2.internal,},FirstTimestamp:2026-06-08 22:47:58.565186905 +0000 UTC m=+0.373798757,LastTimestamp:2026-06-08 22:47:58.565186905 +0000 UTC m=+0.373798757,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-248.ec2.internal,}" Jun 08 22:47:58.567839 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.567828 2567 policy_none.go:47] "Start" Jun 08 22:47:58.568074 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.568056 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-blv6r" Jun 08 22:47:58.599033 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.599015 2567 manager.go:337] "Starting Device Plugin manager" Jun 08 22:47:58.599112 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.599047 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 08 22:47:58.599112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.599057 2567 server.go:87] "Starting device plugin registration server" Jun 08 22:47:58.599333 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.599321 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 08 22:47:58.599407 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.599337 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 08 22:47:58.599571 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.599550 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 08 22:47:58.599727 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.599713 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 08 22:47:58.599782 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.599729 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 08 22:47:58.600737 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.600721 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 08 22:47:58.615660 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.600771 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:58.615660 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.613731 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 08 22:47:58.615660 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.614909 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 08 22:47:58.615660 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.614941 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 08 22:47:58.615660 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.614961 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jun 08 22:47:58.615660 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.614993 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 08 22:47:58.617202 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.617187 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:47:58.699655 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.699634 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.700801 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.700784 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.700890 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.700814 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.700890 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.700825 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.700890 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.700848 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.708725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.708710 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.708785 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.708736 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-248.ec2.internal\": node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:58.715525 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.715475 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal"] Jun 08 22:47:58.715601 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.715536 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.717088 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.717073 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.717162 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.717102 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.717162 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.717112 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.718217 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.718206 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.718349 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.718333 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.718415 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.718364 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.718933 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.718920 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.719000 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.718949 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.719000 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.718964 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.719095 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.718923 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.719095 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.719016 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.719095 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.719025 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.720060 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.720047 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.720095 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.720071 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.721898 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.721876 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.721994 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.721906 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.721994 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.721922 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.722451 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.722437 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:58.743189 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.743162 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-248.ec2.internal\" not found" node="ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.747509 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.747483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/680c3df85c56e803e7a0410016e00c29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal\" (UID: \"680c3df85c56e803e7a0410016e00c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.747604 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.747522 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/680c3df85c56e803e7a0410016e00c29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal\" (UID: \"680c3df85c56e803e7a0410016e00c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.747604 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.747547 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d9b52f40b5c1741b36fc8e18f573c3f1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-248.ec2.internal\" (UID: \"d9b52f40b5c1741b36fc8e18f573c3f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.747696 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.747667 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-248.ec2.internal\" not found" node="ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.823041 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.823001 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:58.847723 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.847699 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/680c3df85c56e803e7a0410016e00c29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal\" (UID: \"680c3df85c56e803e7a0410016e00c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.847793 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.847729 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/680c3df85c56e803e7a0410016e00c29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal\" (UID: \"680c3df85c56e803e7a0410016e00c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.847793 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.847745 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d9b52f40b5c1741b36fc8e18f573c3f1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-248.ec2.internal\" (UID: \"d9b52f40b5c1741b36fc8e18f573c3f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.847854 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.847794 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/680c3df85c56e803e7a0410016e00c29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal\" (UID: \"680c3df85c56e803e7a0410016e00c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.847854 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.847812 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d9b52f40b5c1741b36fc8e18f573c3f1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-248.ec2.internal\" (UID: \"d9b52f40b5c1741b36fc8e18f573c3f1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.847854 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:58.847848 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/680c3df85c56e803e7a0410016e00c29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal\" (UID: \"680c3df85c56e803e7a0410016e00c29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" Jun 08 22:47:58.923838 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:58.923789 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:59.024419 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:59.024328 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:59.047754 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.047730 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" Jun 08 22:47:59.052316 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.052300 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal" Jun 08 22:47:59.125409 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:59.125369 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:59.225877 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:59.225839 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:59.326431 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:59.326321 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:59.426857 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:59.426826 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:59.481310 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.481275 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 08 22:47:59.482014 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.481473 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 22:47:59.510784 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:59.510754 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b52f40b5c1741b36fc8e18f573c3f1.slice/crio-90ffde94649bd271b0941e467d9142ebd357b8d46708c8816e6df4de3a2545e0 WatchSource:0}: Error finding container 90ffde94649bd271b0941e467d9142ebd357b8d46708c8816e6df4de3a2545e0: Status 404 returned error can't find the container with id 90ffde94649bd271b0941e467d9142ebd357b8d46708c8816e6df4de3a2545e0 Jun 08 22:47:59.510993 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:47:59.510980 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680c3df85c56e803e7a0410016e00c29.slice/crio-17cf3a60db692c03527b1e1cfece49ddd56b7de1bc7918f164c6163851ce79b9 WatchSource:0}: Error finding container 17cf3a60db692c03527b1e1cfece49ddd56b7de1bc7918f164c6163851ce79b9: Status 404 returned error can't find the container with id 17cf3a60db692c03527b1e1cfece49ddd56b7de1bc7918f164c6163851ce79b9 Jun 08 22:47:59.516694 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.516676 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 22:47:59.527112 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:47:59.527086 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-248.ec2.internal\" not found" Jun 08 22:47:59.558840 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.558811 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:47:59.570163 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.570131 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-07 22:42:58 +0000 UTC" deadline="2027-11-23 20:07:33.276085793 +0000 UTC" Jun 08 22:47:59.570220 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.570179 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12789h19m33.705921595s" Jun 08 22:47:59.618529 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.618424 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal" event={"ID":"d9b52f40b5c1741b36fc8e18f573c3f1","Type":"ContainerStarted","Data":"90ffde94649bd271b0941e467d9142ebd357b8d46708c8816e6df4de3a2545e0"} Jun 08 22:47:59.619538 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.619517 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" event={"ID":"680c3df85c56e803e7a0410016e00c29","Type":"ContainerStarted","Data":"17cf3a60db692c03527b1e1cfece49ddd56b7de1bc7918f164c6163851ce79b9"} Jun 08 22:47:59.645465 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.645440 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" Jun 08 22:47:59.655828 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.655808 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 22:47:59.658151 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.658137 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal" Jun 08 22:47:59.666097 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.666079 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 22:47:59.743195 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.743166 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:47:59.962841 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:47:59.962812 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:48:00.526401 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.526335 2567 apiserver.go:52] "Watching apiserver" Jun 08 22:48:00.531074 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.531054 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 08 22:48:00.532331 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.532300 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8","openshift-cluster-node-tuning-operator/tuned-kcmw6","openshift-dns/node-resolver-gpjcx","openshift-multus/multus-additional-cni-plugins-djtmb","openshift-multus/multus-xhk8k","openshift-multus/network-metrics-daemon-rds9j","kube-system/konnectivity-agent-54swc","openshift-image-registry/node-ca-c45cn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal","openshift-network-diagnostics/network-check-target-xwkrs","openshift-network-operator/iptables-alerter-5hnxw","openshift-ovn-kubernetes/ovnkube-node-9hb5j","kube-system/global-pull-secret-syncer-xs9jd","kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal"] Jun 08 22:48:00.534258 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.534222 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:00.535306 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.535285 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:00.535427 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.535354 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:00.535959 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.535931 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 08 22:48:00.536046 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.536022 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nkkls\"" Jun 08 22:48:00.536193 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.536178 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 08 22:48:00.536457 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.536433 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.538146 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.537748 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.538146 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.537848 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.538146 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.537912 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2s8nc\"" Jun 08 22:48:00.538146 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.538141 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.539081 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.539051 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.539532 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.539512 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nmb2j\"" Jun 08 22:48:00.539620 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.539531 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.539620 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.539590 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 08 22:48:00.540238 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.540108 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 08 22:48:00.540238 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.540124 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 08 22:48:00.540238 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.540140 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.540238 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.540173 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:00.540497 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.540235 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:00.540684 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.540663 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r6xx4\"" Jun 08 22:48:00.541377 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.541358 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 08 22:48:00.541539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.541453 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.543061 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.542851 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 08 22:48:00.543061 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.542928 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.543270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.543203 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.543345 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.543279 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-62gb9\"" Jun 08 22:48:00.543933 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.543915 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.545312 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.545292 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.545449 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.545430 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 08 22:48:00.545818 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.545673 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 08 22:48:00.545818 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.545687 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.545818 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.545725 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wftb4\"" Jun 08 22:48:00.545818 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.545743 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.546692 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.546672 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:00.546800 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.546747 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:00.547317 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.547063 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8cnvg\"" Jun 08 22:48:00.547317 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.547101 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.547317 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.547118 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.549278 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.549234 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.551405 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.551374 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.551711 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.551693 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8ccx2\"" Jun 08 22:48:00.552777 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.551950 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.552777 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.552016 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.553277 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.553259 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 08 22:48:00.554341 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.554325 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 08 22:48:00.555233 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.555016 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 08 22:48:00.555233 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.555031 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 08 22:48:00.555233 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.555060 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bvtwd\"" Jun 08 22:48:00.555233 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.555134 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 08 22:48:00.555233 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.555210 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.555508 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.555325 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.557616 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557594 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-cni-dir\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.557713 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557638 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-cni-binary-copy\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.557713 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557692 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.557815 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d30efbda-430f-4750-99ba-2da9000234f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.557815 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557748 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d30efbda-430f-4750-99ba-2da9000234f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.557815 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557768 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-sysconfig\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.557815 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557791 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-systemd\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.557815 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557813 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-sysctl-d\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.558026 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557836 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-device-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.558026 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557858 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-run-netns\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558026 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557882 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2987x\" (UniqueName: \"kubernetes.io/projected/b8c10f1b-1d19-4d08-b676-8434b760c799-kube-api-access-2987x\") pod \"node-ca-c45cn\" (UID: \"b8c10f1b-1d19-4d08-b676-8434b760c799\") " pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.558026 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557914 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-run-k8s-cni-cncf-io\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558026 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557937 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-conf-dir\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558026 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557960 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-etc-selinux\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.558026 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.557982 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8s9c\" (UniqueName: \"kubernetes.io/projected/8bef3bb3-12d5-49dd-b50f-11f885052859-kube-api-access-k8s9c\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.558026 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558004 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-dbus\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:00.558026 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558026 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8c10f1b-1d19-4d08-b676-8434b760c799-host\") pod \"node-ca-c45cn\" (UID: \"b8c10f1b-1d19-4d08-b676-8434b760c799\") " pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558049 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-kubernetes\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558070 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-var-lib-kubelet\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558095 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558116 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-hostroot\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558149 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzw5g\" (UniqueName: \"kubernetes.io/projected/f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e-kube-api-access-vzw5g\") pod \"node-resolver-gpjcx\" (UID: \"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e\") " pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558181 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558256 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skt6r\" (UniqueName: \"kubernetes.io/projected/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-kube-api-access-skt6r\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558279 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2946f010-81fb-47ea-a3af-d3605b266d6c-konnectivity-ca\") pod \"konnectivity-agent-54swc\" (UID: \"2946f010-81fb-47ea-a3af-d3605b266d6c\") " pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558309 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d30efbda-430f-4750-99ba-2da9000234f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.558336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558329 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-modprobe-d\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558345 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-socket-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558359 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-sys-fs\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558372 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-sys\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558414 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-socket-dir-parent\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558438 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-var-lib-cni-bin\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558459 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-daemon-config\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558493 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-run-multus-certs\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558520 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-system-cni-dir\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558548 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-sysctl-conf\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558571 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-tuned\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558592 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-kubelet-config\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558615 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-system-cni-dir\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558639 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-registration-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.558672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558661 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-lib-modules\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558682 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-var-lib-kubelet\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558706 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558726 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-host\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558748 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck98c\" (UniqueName: \"kubernetes.io/projected/a9097ddc-23fa-46f4-9568-52ac7aa9abda-kube-api-access-ck98c\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558773 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558814 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e-hosts-file\") pod \"node-resolver-gpjcx\" (UID: \"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e\") " pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558864 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbx9m\" (UniqueName: \"kubernetes.io/projected/d30efbda-430f-4750-99ba-2da9000234f3-kube-api-access-qbx9m\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558891 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd7nf\" (UniqueName: \"kubernetes.io/projected/fbb0f393-3467-4695-8c0f-d31e1f735f9c-kube-api-access-nd7nf\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558916 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9097ddc-23fa-46f4-9568-52ac7aa9abda-tmp\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558946 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-cnibin\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558976 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-os-release\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.558999 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-var-lib-cni-multus\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.559045 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-etc-kubernetes\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.559067 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e-tmp-dir\") pod \"node-resolver-gpjcx\" (UID: \"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e\") " pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.559109 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-cnibin\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.559164 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2946f010-81fb-47ea-a3af-d3605b266d6c-agent-certs\") pod \"konnectivity-agent-54swc\" (UID: \"2946f010-81fb-47ea-a3af-d3605b266d6c\") " pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:00.559270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.559191 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b8c10f1b-1d19-4d08-b676-8434b760c799-serviceca\") pod \"node-ca-c45cn\" (UID: \"b8c10f1b-1d19-4d08-b676-8434b760c799\") " pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.560186 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.559280 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-run\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.560186 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.559309 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-os-release\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.579107 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.579043 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 22:48:00.647082 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.647047 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 08 22:48:00.648302 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.648268 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:48:00.660367 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660342 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-etc-kubernetes\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.660546 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660374 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e-tmp-dir\") pod \"node-resolver-gpjcx\" (UID: \"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e\") " pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.660546 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660416 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-cnibin\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.660546 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660481 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-etc-kubernetes\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.660546 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660517 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2946f010-81fb-47ea-a3af-d3605b266d6c-agent-certs\") pod \"konnectivity-agent-54swc\" (UID: \"2946f010-81fb-47ea-a3af-d3605b266d6c\") " pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:00.660546 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660541 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-cnibin\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.660853 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660554 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b8c10f1b-1d19-4d08-b676-8434b760c799-serviceca\") pod \"node-ca-c45cn\" (UID: \"b8c10f1b-1d19-4d08-b676-8434b760c799\") " pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.660853 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660591 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-run\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.660853 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660621 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-slash\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.660853 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660672 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-cni-bin\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.660853 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660714 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e-tmp-dir\") pod \"node-resolver-gpjcx\" (UID: \"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e\") " pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.660853 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660722 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-os-release\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.660853 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660820 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-run\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.660853 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660836 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-os-release\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660759 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-run-systemd\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660905 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-var-lib-openvswitch\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660898 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660956 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-run-openvswitch\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.660982 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-cni-dir\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661029 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-cni-binary-copy\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661073 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661099 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d30efbda-430f-4750-99ba-2da9000234f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661126 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b8c10f1b-1d19-4d08-b676-8434b760c799-serviceca\") pod \"node-ca-c45cn\" (UID: \"b8c10f1b-1d19-4d08-b676-8434b760c799\") " pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661135 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-cni-dir\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.661173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661126 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d30efbda-430f-4750-99ba-2da9000234f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661200 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-sysconfig\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661229 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-systemd\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661255 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0446f325-1d19-4795-bec4-92298eaea1bf-iptables-alerter-script\") pod \"iptables-alerter-5hnxw\" (UID: \"0446f325-1d19-4795-bec4-92298eaea1bf\") " pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661279 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-sysctl-d\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661304 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-etc-openvswitch\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661310 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661328 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzdgh\" (UniqueName: \"kubernetes.io/projected/1b1f88ae-6de8-4d13-9f67-aa38624ff076-kube-api-access-kzdgh\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661365 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0446f325-1d19-4795-bec4-92298eaea1bf-host-slash\") pod \"iptables-alerter-5hnxw\" (UID: \"0446f325-1d19-4795-bec4-92298eaea1bf\") " pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661402 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-device-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661426 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-run-netns\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661450 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2987x\" (UniqueName: \"kubernetes.io/projected/b8c10f1b-1d19-4d08-b676-8434b760c799-kube-api-access-2987x\") pod \"node-ca-c45cn\" (UID: \"b8c10f1b-1d19-4d08-b676-8434b760c799\") " pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661474 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-run-ovn\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661497 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1b1f88ae-6de8-4d13-9f67-aa38624ff076-ovnkube-script-lib\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661522 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-run-k8s-cni-cncf-io\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-conf-dir\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661573 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-etc-selinux\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.661700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661596 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8s9c\" (UniqueName: \"kubernetes.io/projected/8bef3bb3-12d5-49dd-b50f-11f885052859-kube-api-access-k8s9c\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661620 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-run-netns\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661643 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-cni-binary-copy\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661700 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-sysconfig\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661750 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-systemd\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661789 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-dbus\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661648 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-dbus\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661799 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d30efbda-430f-4750-99ba-2da9000234f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661825 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8c10f1b-1d19-4d08-b676-8434b760c799-host\") pod \"node-ca-c45cn\" (UID: \"b8c10f1b-1d19-4d08-b676-8434b760c799\") " pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661851 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-kubernetes\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661889 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-device-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661877 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-var-lib-kubelet\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661893 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-sysctl-d\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661896 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d30efbda-430f-4750-99ba-2da9000234f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661929 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-node-log\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661930 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8c10f1b-1d19-4d08-b676-8434b760c799-host\") pod \"node-ca-c45cn\" (UID: \"b8c10f1b-1d19-4d08-b676-8434b760c799\") " pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.661995 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-kubernetes\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.662549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662015 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-var-lib-kubelet\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662051 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-run-k8s-cni-cncf-io\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662052 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-conf-dir\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662079 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1b1f88ae-6de8-4d13-9f67-aa38624ff076-ovnkube-config\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662105 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662130 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-hostroot\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662130 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-etc-selinux\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.662217 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.662290 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret podName:c44e313d-1c49-4766-9a6b-daa3e68b9c2e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:01.162259632 +0000 UTC m=+2.970871495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret") pod "global-pull-secret-syncer-xs9jd" (UID: "c44e313d-1c49-4766-9a6b-daa3e68b9c2e") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662319 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzw5g\" (UniqueName: \"kubernetes.io/projected/f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e-kube-api-access-vzw5g\") pod \"node-resolver-gpjcx\" (UID: \"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e\") " pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662356 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662406 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-hostroot\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662384 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-log-socket\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skt6r\" (UniqueName: \"kubernetes.io/projected/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-kube-api-access-skt6r\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662448 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-run-netns\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662471 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2946f010-81fb-47ea-a3af-d3605b266d6c-konnectivity-ca\") pod \"konnectivity-agent-54swc\" (UID: \"2946f010-81fb-47ea-a3af-d3605b266d6c\") " pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662541 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d30efbda-430f-4750-99ba-2da9000234f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.663357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662571 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-systemd-units\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662597 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-modprobe-d\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662622 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-run-ovn-kubernetes\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662648 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-socket-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662672 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-sys-fs\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662696 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-sys\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662720 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-cni-netd\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662747 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhjgv\" (UniqueName: \"kubernetes.io/projected/0446f325-1d19-4795-bec4-92298eaea1bf-kube-api-access-bhjgv\") pod \"iptables-alerter-5hnxw\" (UID: \"0446f325-1d19-4795-bec4-92298eaea1bf\") " pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662753 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-modprobe-d\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662772 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-socket-dir-parent\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662814 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-var-lib-cni-bin\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662847 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-daemon-config\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662876 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-run-multus-certs\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662895 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-socket-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662902 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-system-cni-dir\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662930 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-sysctl-conf\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662968 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-sys-fs\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.664173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662978 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2946f010-81fb-47ea-a3af-d3605b266d6c-konnectivity-ca\") pod \"konnectivity-agent-54swc\" (UID: \"2946f010-81fb-47ea-a3af-d3605b266d6c\") " pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.662970 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-tuned\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663020 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-sys\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663021 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-kubelet-config\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663058 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-system-cni-dir\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663063 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-kubelet-config\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663085 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-registration-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663111 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-lib-modules\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663139 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-system-cni-dir\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663139 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-kubelet\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663171 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1b1f88ae-6de8-4d13-9f67-aa38624ff076-env-overrides\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663197 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-var-lib-kubelet\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663223 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-host\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663283 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck98c\" (UniqueName: \"kubernetes.io/projected/a9097ddc-23fa-46f4-9568-52ac7aa9abda-kube-api-access-ck98c\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663287 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-lib-modules\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663308 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.665050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663338 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663365 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e-hosts-file\") pod \"node-resolver-gpjcx\" (UID: \"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e\") " pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663382 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-socket-dir-parent\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663413 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d30efbda-430f-4750-99ba-2da9000234f3-system-cni-dir\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663455 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-run-multus-certs\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663500 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-var-lib-cni-bin\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663197 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d30efbda-430f-4750-99ba-2da9000234f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663520 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-sysctl-conf\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663339 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-var-lib-kubelet\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663583 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-multus-daemon-config\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663597 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.663602 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663340 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bef3bb3-12d5-49dd-b50f-11f885052859-registration-dir\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.663653 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs podName:fbb0f393-3467-4695-8c0f-d31e1f735f9c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:01.163635897 +0000 UTC m=+2.972247730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs") pod "network-metrics-daemon-rds9j" (UID: "fbb0f393-3467-4695-8c0f-d31e1f735f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663649 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9097ddc-23fa-46f4-9568-52ac7aa9abda-host\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663675 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e-hosts-file\") pod \"node-resolver-gpjcx\" (UID: \"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e\") " pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663823 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbx9m\" (UniqueName: \"kubernetes.io/projected/d30efbda-430f-4750-99ba-2da9000234f3-kube-api-access-qbx9m\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.665883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663868 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd7nf\" (UniqueName: \"kubernetes.io/projected/fbb0f393-3467-4695-8c0f-d31e1f735f9c-kube-api-access-nd7nf\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663894 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9097ddc-23fa-46f4-9568-52ac7aa9abda-tmp\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663918 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b1f88ae-6de8-4d13-9f67-aa38624ff076-ovn-node-metrics-cert\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663959 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-cnibin\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.663990 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-os-release\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.664014 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-var-lib-cni-multus\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.664086 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-cnibin\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.664138 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-host-var-lib-cni-multus\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.664143 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-os-release\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.664808 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2946f010-81fb-47ea-a3af-d3605b266d6c-agent-certs\") pod \"konnectivity-agent-54swc\" (UID: \"2946f010-81fb-47ea-a3af-d3605b266d6c\") " pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.666224 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9097ddc-23fa-46f4-9568-52ac7aa9abda-tmp\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.666482 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.666452 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9097ddc-23fa-46f4-9568-52ac7aa9abda-etc-tuned\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.675559 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.675537 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:00.675685 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.675564 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:00.675685 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.675586 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6dlkj for pod openshift-network-diagnostics/network-check-target-xwkrs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:00.675685 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.675537 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2987x\" (UniqueName: \"kubernetes.io/projected/b8c10f1b-1d19-4d08-b676-8434b760c799-kube-api-access-2987x\") pod \"node-ca-c45cn\" (UID: \"b8c10f1b-1d19-4d08-b676-8434b760c799\") " pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.675685 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:00.675656 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj podName:49b1add5-a152-4d44-81f9-a7b7e4d8d12a nodeName:}" failed. No retries permitted until 2026-06-08 22:48:01.175631152 +0000 UTC m=+2.984243008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6dlkj" (UniqueName: "kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj") pod "network-check-target-xwkrs" (UID: "49b1add5-a152-4d44-81f9-a7b7e4d8d12a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:00.676539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.676506 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck98c\" (UniqueName: \"kubernetes.io/projected/a9097ddc-23fa-46f4-9568-52ac7aa9abda-kube-api-access-ck98c\") pod \"tuned-kcmw6\" (UID: \"a9097ddc-23fa-46f4-9568-52ac7aa9abda\") " pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.677240 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.677215 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8s9c\" (UniqueName: \"kubernetes.io/projected/8bef3bb3-12d5-49dd-b50f-11f885052859-kube-api-access-k8s9c\") pod \"aws-ebs-csi-driver-node-vp4n8\" (UID: \"8bef3bb3-12d5-49dd-b50f-11f885052859\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.677822 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.677792 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbx9m\" (UniqueName: \"kubernetes.io/projected/d30efbda-430f-4750-99ba-2da9000234f3-kube-api-access-qbx9m\") pod \"multus-additional-cni-plugins-djtmb\" (UID: \"d30efbda-430f-4750-99ba-2da9000234f3\") " pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.678154 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.678139 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd7nf\" (UniqueName: \"kubernetes.io/projected/fbb0f393-3467-4695-8c0f-d31e1f735f9c-kube-api-access-nd7nf\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:00.678210 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.678176 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzw5g\" (UniqueName: \"kubernetes.io/projected/f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e-kube-api-access-vzw5g\") pod \"node-resolver-gpjcx\" (UID: \"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e\") " pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.678278 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.678206 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skt6r\" (UniqueName: \"kubernetes.io/projected/260a4693-1d70-4eea-ac68-9bf0ed2c87d9-kube-api-access-skt6r\") pod \"multus-xhk8k\" (UID: \"260a4693-1d70-4eea-ac68-9bf0ed2c87d9\") " pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.764543 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764507 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-run-ovn-kubernetes\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.764725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764554 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-cni-netd\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.764725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764577 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhjgv\" (UniqueName: \"kubernetes.io/projected/0446f325-1d19-4795-bec4-92298eaea1bf-kube-api-access-bhjgv\") pod \"iptables-alerter-5hnxw\" (UID: \"0446f325-1d19-4795-bec4-92298eaea1bf\") " pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.764725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764597 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-kubelet\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.764725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764614 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1b1f88ae-6de8-4d13-9f67-aa38624ff076-env-overrides\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.764725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764626 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-run-ovn-kubernetes\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.764725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764645 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.764725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764625 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-cni-netd\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.764725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764669 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-kubelet\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.764725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764693 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764737 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b1f88ae-6de8-4d13-9f67-aa38624ff076-ovn-node-metrics-cert\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764788 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-slash\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764806 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-cni-bin\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764823 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-run-systemd\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764846 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-var-lib-openvswitch\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764869 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-run-openvswitch\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764898 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0446f325-1d19-4795-bec4-92298eaea1bf-iptables-alerter-script\") pod \"iptables-alerter-5hnxw\" (UID: \"0446f325-1d19-4795-bec4-92298eaea1bf\") " pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764905 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-run-systemd\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764925 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-etc-openvswitch\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764931 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-slash\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764895 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-cni-bin\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764960 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-var-lib-openvswitch\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764967 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-run-openvswitch\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.764991 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-etc-openvswitch\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765007 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzdgh\" (UniqueName: \"kubernetes.io/projected/1b1f88ae-6de8-4d13-9f67-aa38624ff076-kube-api-access-kzdgh\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765031 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0446f325-1d19-4795-bec4-92298eaea1bf-host-slash\") pod \"iptables-alerter-5hnxw\" (UID: \"0446f325-1d19-4795-bec4-92298eaea1bf\") " pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.765160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765058 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-run-ovn\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765097 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0446f325-1d19-4795-bec4-92298eaea1bf-host-slash\") pod \"iptables-alerter-5hnxw\" (UID: \"0446f325-1d19-4795-bec4-92298eaea1bf\") " pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765100 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-run-ovn\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765128 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1b1f88ae-6de8-4d13-9f67-aa38624ff076-ovnkube-script-lib\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765158 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-run-netns\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-node-log\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765193 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1b1f88ae-6de8-4d13-9f67-aa38624ff076-env-overrides\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765214 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1b1f88ae-6de8-4d13-9f67-aa38624ff076-ovnkube-config\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765221 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-host-run-netns\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765260 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-log-socket\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765294 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-systemd-units\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765259 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-node-log\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765321 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-log-socket\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765364 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1b1f88ae-6de8-4d13-9f67-aa38624ff076-systemd-units\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765476 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0446f325-1d19-4795-bec4-92298eaea1bf-iptables-alerter-script\") pod \"iptables-alerter-5hnxw\" (UID: \"0446f325-1d19-4795-bec4-92298eaea1bf\") " pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765677 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1b1f88ae-6de8-4d13-9f67-aa38624ff076-ovnkube-script-lib\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.765825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.765786 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1b1f88ae-6de8-4d13-9f67-aa38624ff076-ovnkube-config\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.767191 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.767168 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b1f88ae-6de8-4d13-9f67-aa38624ff076-ovn-node-metrics-cert\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.772780 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.772725 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhjgv\" (UniqueName: \"kubernetes.io/projected/0446f325-1d19-4795-bec4-92298eaea1bf-kube-api-access-bhjgv\") pod \"iptables-alerter-5hnxw\" (UID: \"0446f325-1d19-4795-bec4-92298eaea1bf\") " pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.772902 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.772789 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzdgh\" (UniqueName: \"kubernetes.io/projected/1b1f88ae-6de8-4d13-9f67-aa38624ff076-kube-api-access-kzdgh\") pod \"ovnkube-node-9hb5j\" (UID: \"1b1f88ae-6de8-4d13-9f67-aa38624ff076\") " pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:00.846439 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.846354 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:00.855229 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.855204 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gpjcx" Jun 08 22:48:00.863427 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.863388 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-djtmb" Jun 08 22:48:00.870159 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.870139 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xhk8k" Jun 08 22:48:00.877499 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.877478 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c45cn" Jun 08 22:48:00.884158 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.884136 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" Jun 08 22:48:00.889797 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.889777 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" Jun 08 22:48:00.897303 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.897283 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5hnxw" Jun 08 22:48:00.902456 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:00.902435 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:01.168205 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.168173 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:01.168380 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.168243 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:01.168380 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:01.168344 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:01.168380 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:01.168363 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:01.168546 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:01.168436 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs podName:fbb0f393-3467-4695-8c0f-d31e1f735f9c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:02.168415737 +0000 UTC m=+3.977027572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs") pod "network-metrics-daemon-rds9j" (UID: "fbb0f393-3467-4695-8c0f-d31e1f735f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:01.168546 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:01.168460 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret podName:c44e313d-1c49-4766-9a6b-daa3e68b9c2e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:02.168450447 +0000 UTC m=+3.977062286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret") pod "global-pull-secret-syncer-xs9jd" (UID: "c44e313d-1c49-4766-9a6b-daa3e68b9c2e") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:01.268999 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.268964 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:01.269155 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:01.269113 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:01.269155 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:01.269130 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:01.269155 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:01.269139 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6dlkj for pod openshift-network-diagnostics/network-check-target-xwkrs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:01.269330 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:01.269190 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj podName:49b1add5-a152-4d44-81f9-a7b7e4d8d12a nodeName:}" failed. No retries permitted until 2026-06-08 22:48:02.269173861 +0000 UTC m=+4.077785716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dlkj" (UniqueName: "kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj") pod "network-check-target-xwkrs" (UID: "49b1add5-a152-4d44-81f9-a7b7e4d8d12a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:01.355593 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:01.355536 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ab127e_d27e_4006_93f8_bb2ba3ce2c6e.slice/crio-0c9ad0f16130187cfde4c57fceb281872ad04ec641fc7877708a10c44f127e4a WatchSource:0}: Error finding container 0c9ad0f16130187cfde4c57fceb281872ad04ec641fc7877708a10c44f127e4a: Status 404 returned error can't find the container with id 0c9ad0f16130187cfde4c57fceb281872ad04ec641fc7877708a10c44f127e4a Jun 08 22:48:01.356733 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:01.356710 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bef3bb3_12d5_49dd_b50f_11f885052859.slice/crio-096c2f0185df1ab257f856abaebb6bb2f5bf49df05124e5e43554dafa361ee16 WatchSource:0}: Error finding container 096c2f0185df1ab257f856abaebb6bb2f5bf49df05124e5e43554dafa361ee16: Status 404 returned error can't find the container with id 096c2f0185df1ab257f856abaebb6bb2f5bf49df05124e5e43554dafa361ee16 Jun 08 22:48:01.358066 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:01.358044 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd30efbda_430f_4750_99ba_2da9000234f3.slice/crio-e135c645ff9772ae8d38e9266eb65a6949da60b9b40ece2ec72287ae2a24f0a5 WatchSource:0}: Error finding container e135c645ff9772ae8d38e9266eb65a6949da60b9b40ece2ec72287ae2a24f0a5: Status 404 returned error can't find the container with id e135c645ff9772ae8d38e9266eb65a6949da60b9b40ece2ec72287ae2a24f0a5 Jun 08 22:48:01.360244 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:01.360212 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2946f010_81fb_47ea_a3af_d3605b266d6c.slice/crio-3768259ac6aec100cf370fd06fe84fe245a38722e443c715f33dc13962a18b88 WatchSource:0}: Error finding container 3768259ac6aec100cf370fd06fe84fe245a38722e443c715f33dc13962a18b88: Status 404 returned error can't find the container with id 3768259ac6aec100cf370fd06fe84fe245a38722e443c715f33dc13962a18b88 Jun 08 22:48:01.362550 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:01.362505 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c10f1b_1d19_4d08_b676_8434b760c799.slice/crio-37619edfe0408b1634575e404af62f61562115ed6880ff1ca87b1ecd94a52959 WatchSource:0}: Error finding container 37619edfe0408b1634575e404af62f61562115ed6880ff1ca87b1ecd94a52959: Status 404 returned error can't find the container with id 37619edfe0408b1634575e404af62f61562115ed6880ff1ca87b1ecd94a52959 Jun 08 22:48:01.363716 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:01.363686 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b1f88ae_6de8_4d13_9f67_aa38624ff076.slice/crio-c80b81551829fca99410346da90c64f386df9dea30465875fc8fd984c1e60765 WatchSource:0}: Error finding container c80b81551829fca99410346da90c64f386df9dea30465875fc8fd984c1e60765: Status 404 returned error can't find the container with id c80b81551829fca99410346da90c64f386df9dea30465875fc8fd984c1e60765 Jun 08 22:48:01.364806 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:01.364779 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0446f325_1d19_4795_bec4_92298eaea1bf.slice/crio-5f034aa954d28ada424dea4d9f341f2d869edd76a877c4c6a76756cef45258be WatchSource:0}: Error finding container 5f034aa954d28ada424dea4d9f341f2d869edd76a877c4c6a76756cef45258be: Status 404 returned error can't find the container with id 5f034aa954d28ada424dea4d9f341f2d869edd76a877c4c6a76756cef45258be Jun 08 22:48:01.365976 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:01.365944 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260a4693_1d70_4eea_ac68_9bf0ed2c87d9.slice/crio-38d999378e8ca83c1194e6ca0c8f63d6b99d25f85bbdf18e14f103d52442d8f1 WatchSource:0}: Error finding container 38d999378e8ca83c1194e6ca0c8f63d6b99d25f85bbdf18e14f103d52442d8f1: Status 404 returned error can't find the container with id 38d999378e8ca83c1194e6ca0c8f63d6b99d25f85bbdf18e14f103d52442d8f1 Jun 08 22:48:01.367443 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:01.367387 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9097ddc_23fa_46f4_9568_52ac7aa9abda.slice/crio-7eeebff64ce50b849f26e8fe7d2b88d8a3248d20f90def17d375260dd8df8a13 WatchSource:0}: Error finding container 7eeebff64ce50b849f26e8fe7d2b88d8a3248d20f90def17d375260dd8df8a13: Status 404 returned error can't find the container with id 7eeebff64ce50b849f26e8fe7d2b88d8a3248d20f90def17d375260dd8df8a13 Jun 08 22:48:01.624102 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.623920 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gpjcx" event={"ID":"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e","Type":"ContainerStarted","Data":"0c9ad0f16130187cfde4c57fceb281872ad04ec641fc7877708a10c44f127e4a"} Jun 08 22:48:01.625755 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.625727 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" event={"ID":"a9097ddc-23fa-46f4-9568-52ac7aa9abda","Type":"ContainerStarted","Data":"7eeebff64ce50b849f26e8fe7d2b88d8a3248d20f90def17d375260dd8df8a13"} Jun 08 22:48:01.628838 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.628809 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xhk8k" event={"ID":"260a4693-1d70-4eea-ac68-9bf0ed2c87d9","Type":"ContainerStarted","Data":"38d999378e8ca83c1194e6ca0c8f63d6b99d25f85bbdf18e14f103d52442d8f1"} Jun 08 22:48:01.629962 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.629932 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c45cn" event={"ID":"b8c10f1b-1d19-4d08-b676-8434b760c799","Type":"ContainerStarted","Data":"37619edfe0408b1634575e404af62f61562115ed6880ff1ca87b1ecd94a52959"} Jun 08 22:48:01.631722 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.631696 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal" event={"ID":"d9b52f40b5c1741b36fc8e18f573c3f1","Type":"ContainerStarted","Data":"8bb98191b729c999031a423a2b58fc200eb98f7e569b441c204fd748f33cda32"} Jun 08 22:48:01.633139 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.633120 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" event={"ID":"680c3df85c56e803e7a0410016e00c29","Type":"ContainerStarted","Data":"af06b3ce650790cc38c2a0ebf0f2108ff3af35e6cff117123058bbd940060bc4"} Jun 08 22:48:01.634923 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.634897 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5hnxw" event={"ID":"0446f325-1d19-4795-bec4-92298eaea1bf","Type":"ContainerStarted","Data":"5f034aa954d28ada424dea4d9f341f2d869edd76a877c4c6a76756cef45258be"} Jun 08 22:48:01.635993 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.635974 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerStarted","Data":"c80b81551829fca99410346da90c64f386df9dea30465875fc8fd984c1e60765"} Jun 08 22:48:01.637122 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.637102 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-54swc" event={"ID":"2946f010-81fb-47ea-a3af-d3605b266d6c","Type":"ContainerStarted","Data":"3768259ac6aec100cf370fd06fe84fe245a38722e443c715f33dc13962a18b88"} Jun 08 22:48:01.641113 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.638635 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-djtmb" event={"ID":"d30efbda-430f-4750-99ba-2da9000234f3","Type":"ContainerStarted","Data":"e135c645ff9772ae8d38e9266eb65a6949da60b9b40ece2ec72287ae2a24f0a5"} Jun 08 22:48:01.642353 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.642332 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" event={"ID":"8bef3bb3-12d5-49dd-b50f-11f885052859","Type":"ContainerStarted","Data":"096c2f0185df1ab257f856abaebb6bb2f5bf49df05124e5e43554dafa361ee16"} Jun 08 22:48:01.655961 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.655908 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-248.ec2.internal" podStartSLOduration=2.655888922 podStartE2EDuration="2.655888922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:01.644563974 +0000 UTC m=+3.453175831" watchObservedRunningTime="2026-06-08 22:48:01.655888922 +0000 UTC m=+3.464500779" Jun 08 22:48:01.910981 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.910938 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9lvt2" Jun 08 22:48:01.918596 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:01.918564 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9lvt2" Jun 08 22:48:02.176246 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.176206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:02.176472 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.176281 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:02.176472 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.176452 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:02.176622 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.176513 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs podName:fbb0f393-3467-4695-8c0f-d31e1f735f9c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:04.176494606 +0000 UTC m=+5.985106443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs") pod "network-metrics-daemon-rds9j" (UID: "fbb0f393-3467-4695-8c0f-d31e1f735f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:02.176913 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.176895 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:02.176974 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.176949 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret podName:c44e313d-1c49-4766-9a6b-daa3e68b9c2e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:04.176934321 +0000 UTC m=+5.985546153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret") pod "global-pull-secret-syncer-xs9jd" (UID: "c44e313d-1c49-4766-9a6b-daa3e68b9c2e") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:02.276978 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.276939 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:02.277166 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.277103 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:02.277166 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.277124 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:02.277166 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.277137 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6dlkj for pod openshift-network-diagnostics/network-check-target-xwkrs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:02.277315 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.277195 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj podName:49b1add5-a152-4d44-81f9-a7b7e4d8d12a nodeName:}" failed. No retries permitted until 2026-06-08 22:48:04.277177332 +0000 UTC m=+6.085789179 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dlkj" (UniqueName: "kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj") pod "network-check-target-xwkrs" (UID: "49b1add5-a152-4d44-81f9-a7b7e4d8d12a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:02.618189 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.617606 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:02.618189 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.617753 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:02.618189 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.617746 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:02.618189 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.617853 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:02.618189 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.617903 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:02.618189 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:02.617970 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:02.654797 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.654767 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 22:48:02.656672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.656642 2567 generic.go:356] "Generic (PLEG): container finished" podID="680c3df85c56e803e7a0410016e00c29" containerID="af06b3ce650790cc38c2a0ebf0f2108ff3af35e6cff117123058bbd940060bc4" exitCode=0 Jun 08 22:48:02.656672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.656675 2567 generic.go:356] "Generic (PLEG): container finished" podID="680c3df85c56e803e7a0410016e00c29" containerID="f9f161486d833683cfeb15d5a30943e27d63f4f58d13d1faed8167873b93e973" exitCode=1 Jun 08 22:48:02.657470 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.657451 2567 scope.go:117] "RemoveContainer" containerID="f9f161486d833683cfeb15d5a30943e27d63f4f58d13d1faed8167873b93e973" Jun 08 22:48:02.657707 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.657678 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" event={"ID":"680c3df85c56e803e7a0410016e00c29","Type":"ContainerDied","Data":"af06b3ce650790cc38c2a0ebf0f2108ff3af35e6cff117123058bbd940060bc4"} Jun 08 22:48:02.657788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.657756 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" event={"ID":"680c3df85c56e803e7a0410016e00c29","Type":"ContainerDied","Data":"f9f161486d833683cfeb15d5a30943e27d63f4f58d13d1faed8167873b93e973"} Jun 08 22:48:02.922005 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.921592 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 22:43:01 +0000 UTC" deadline="2027-11-23 20:48:38.004060033 +0000 UTC" Jun 08 22:48:02.922005 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:02.921626 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12790h0m35.082437784s" Jun 08 22:48:03.670149 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:03.670121 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 22:48:03.672105 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:03.672075 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" event={"ID":"680c3df85c56e803e7a0410016e00c29","Type":"ContainerStarted","Data":"c01b6364562fc3c0b7a33b386b1df20061d30dc41f72f5cf4781ad66bee0e0ae"} Jun 08 22:48:03.922303 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:03.922202 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 22:43:01 +0000 UTC" deadline="2028-03-05 20:30:55.225355728 +0000 UTC" Jun 08 22:48:03.922303 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:03.922244 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15261h42m51.303116024s" Jun 08 22:48:04.195807 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:04.195720 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:04.195961 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:04.195831 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:04.196032 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.195990 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:04.196090 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.196054 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret podName:c44e313d-1c49-4766-9a6b-daa3e68b9c2e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:08.196035014 +0000 UTC m=+10.004646848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret") pod "global-pull-secret-syncer-xs9jd" (UID: "c44e313d-1c49-4766-9a6b-daa3e68b9c2e") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:04.196577 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.196558 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:04.196653 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.196618 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs podName:fbb0f393-3467-4695-8c0f-d31e1f735f9c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:08.196601346 +0000 UTC m=+10.005213185 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs") pod "network-metrics-daemon-rds9j" (UID: "fbb0f393-3467-4695-8c0f-d31e1f735f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:04.296294 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:04.296257 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:04.296500 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.296450 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:04.296500 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.296471 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:04.296500 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.296485 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6dlkj for pod openshift-network-diagnostics/network-check-target-xwkrs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:04.296647 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.296549 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj podName:49b1add5-a152-4d44-81f9-a7b7e4d8d12a nodeName:}" failed. No retries permitted until 2026-06-08 22:48:08.296529654 +0000 UTC m=+10.105141506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dlkj" (UniqueName: "kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj") pod "network-check-target-xwkrs" (UID: "49b1add5-a152-4d44-81f9-a7b7e4d8d12a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:04.618461 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:04.617633 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:04.618461 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.617772 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:04.618461 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:04.618159 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:04.618461 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.618262 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:04.618461 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:04.618299 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:04.618461 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:04.618372 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:06.615531 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:06.615502 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:06.615971 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:06.615499 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:06.615971 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:06.615647 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:06.615971 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:06.615499 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:06.615971 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:06.615695 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:06.615971 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:06.615793 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:08.228789 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:08.228738 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:08.229245 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:08.228813 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:08.229245 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.228954 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:08.229245 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.228956 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:08.229245 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.229018 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs podName:fbb0f393-3467-4695-8c0f-d31e1f735f9c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:16.228997478 +0000 UTC m=+18.037609315 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs") pod "network-metrics-daemon-rds9j" (UID: "fbb0f393-3467-4695-8c0f-d31e1f735f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:08.229245 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.229035 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret podName:c44e313d-1c49-4766-9a6b-daa3e68b9c2e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:16.229026838 +0000 UTC m=+18.037638677 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret") pod "global-pull-secret-syncer-xs9jd" (UID: "c44e313d-1c49-4766-9a6b-daa3e68b9c2e") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:08.329476 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:08.329434 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:08.329659 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.329614 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:08.329659 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.329636 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:08.329659 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.329650 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6dlkj for pod openshift-network-diagnostics/network-check-target-xwkrs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:08.329831 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.329714 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj podName:49b1add5-a152-4d44-81f9-a7b7e4d8d12a nodeName:}" failed. No retries permitted until 2026-06-08 22:48:16.329696695 +0000 UTC m=+18.138308541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dlkj" (UniqueName: "kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj") pod "network-check-target-xwkrs" (UID: "49b1add5-a152-4d44-81f9-a7b7e4d8d12a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:08.616988 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:08.616909 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:08.617135 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:08.617024 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:08.617135 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.617075 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:08.617641 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.617614 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:08.617770 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:08.617669 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:08.617770 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:08.617746 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:10.615531 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:10.615446 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:10.616113 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:10.615447 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:10.616113 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:10.615589 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:10.616113 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:10.615584 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:10.616113 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:10.615684 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:10.616113 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:10.615770 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:12.615975 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:12.615939 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:12.616449 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:12.615939 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:12.616449 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:12.616075 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:12.616449 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:12.615940 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:12.616449 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:12.616149 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:12.616449 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:12.616220 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:14.616296 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:14.616265 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:14.616799 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:14.616262 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:14.616799 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:14.616406 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:14.616799 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:14.616418 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:14.616799 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:14.616498 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:14.616799 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:14.616598 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:16.290008 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:16.289967 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:16.290491 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:16.290028 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:16.290491 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.290127 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:16.290491 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.290142 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:16.290491 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.290198 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret podName:c44e313d-1c49-4766-9a6b-daa3e68b9c2e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:32.290181841 +0000 UTC m=+34.098793673 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret") pod "global-pull-secret-syncer-xs9jd" (UID: "c44e313d-1c49-4766-9a6b-daa3e68b9c2e") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:16.290491 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.290214 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs podName:fbb0f393-3467-4695-8c0f-d31e1f735f9c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:32.290206767 +0000 UTC m=+34.098818599 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs") pod "network-metrics-daemon-rds9j" (UID: "fbb0f393-3467-4695-8c0f-d31e1f735f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:16.390706 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:16.390668 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:16.390869 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.390814 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:16.390869 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.390841 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:16.390869 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.390856 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6dlkj for pod openshift-network-diagnostics/network-check-target-xwkrs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:16.390986 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.390928 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj podName:49b1add5-a152-4d44-81f9-a7b7e4d8d12a nodeName:}" failed. No retries permitted until 2026-06-08 22:48:32.390902227 +0000 UTC m=+34.199514085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dlkj" (UniqueName: "kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj") pod "network-check-target-xwkrs" (UID: "49b1add5-a152-4d44-81f9-a7b7e4d8d12a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:16.615470 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:16.615376 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:16.615470 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:16.615423 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:16.615662 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:16.615375 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:16.615662 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.615509 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:16.615662 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.615562 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:16.615662 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:16.615655 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:18.616915 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:18.616882 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:18.617384 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:18.616977 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:18.617384 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:18.616990 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:18.617384 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:18.617099 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:18.617384 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:18.617085 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:18.617384 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:18.617245 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:20.615946 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.615635 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:20.616605 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.615634 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:20.616605 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:20.616047 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:20.616605 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:20.616147 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:20.616605 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.615633 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:20.616605 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:20.616262 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:20.707445 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.707407 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c45cn" event={"ID":"b8c10f1b-1d19-4d08-b676-8434b760c799","Type":"ContainerStarted","Data":"016a4f043d2d198f3a127227bf9c22fd4fa7b61b8c34bb77d5d5f1ba4a34b3bd"} Jun 08 22:48:20.711475 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.711424 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 22:48:20.711918 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.711889 2567 generic.go:356] "Generic (PLEG): container finished" podID="1b1f88ae-6de8-4d13-9f67-aa38624ff076" containerID="ac20e063daf05fca65cd2fd6c0b2356701ea2b15ee3f6436e5bf34af6e98a3f3" exitCode=1 Jun 08 22:48:20.712058 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.711963 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerStarted","Data":"2c72b465f6b449154ec90332eb284b1daeb09cb44aa6e39e6ca5ee2535c78224"} Jun 08 22:48:20.712058 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.712052 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerStarted","Data":"bab6e12ca845e8f5d4280b413f7335b8e86c472427fd0046a371f4d95140cad3"} Jun 08 22:48:20.712166 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.712108 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerStarted","Data":"f4aedf0888ea5407a2ce9cf348d2b372916e82ef6c8129108beec66785a88406"} Jun 08 22:48:20.712166 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.712157 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerStarted","Data":"e72de8efc5dce81db41e97603c98e2c0d2ce21528e3f4b9e38fd829dbcba326d"} Jun 08 22:48:20.712262 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.712205 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerDied","Data":"ac20e063daf05fca65cd2fd6c0b2356701ea2b15ee3f6436e5bf34af6e98a3f3"} Jun 08 22:48:20.712262 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.712255 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerStarted","Data":"4adc578059baf56158b4f105d5ec786ac16e07c27062186e012afb6c5c2112b8"} Jun 08 22:48:20.713784 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.713762 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-54swc" event={"ID":"2946f010-81fb-47ea-a3af-d3605b266d6c","Type":"ContainerStarted","Data":"c9a3867fa211c25b58955be45f72d77d37ce4e37531f4faaf4f44ac35bccb2cf"} Jun 08 22:48:20.715654 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.715628 2567 generic.go:356] "Generic (PLEG): container finished" podID="d30efbda-430f-4750-99ba-2da9000234f3" containerID="b5768d24749f4d226220c742a9e1684d1b27eb26bbde03af71059df190776ec4" exitCode=0 Jun 08 22:48:20.715782 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.715706 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-djtmb" event={"ID":"d30efbda-430f-4750-99ba-2da9000234f3","Type":"ContainerDied","Data":"b5768d24749f4d226220c742a9e1684d1b27eb26bbde03af71059df190776ec4"} Jun 08 22:48:20.721952 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.720677 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal" podStartSLOduration=21.720660542 podStartE2EDuration="21.720660542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:03.686605751 +0000 UTC m=+5.495217608" watchObservedRunningTime="2026-06-08 22:48:20.720660542 +0000 UTC m=+22.529272402" Jun 08 22:48:20.723503 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.723478 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" event={"ID":"8bef3bb3-12d5-49dd-b50f-11f885052859","Type":"ContainerStarted","Data":"0ae54866dd2bc90165edfe1181c3c2d45ed48d11da958b20f8c58862bf419a55"} Jun 08 22:48:20.726072 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.726005 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gpjcx" event={"ID":"f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e","Type":"ContainerStarted","Data":"803d8815ba395d2143671046ee8cce9886a07c4566a21671b6b7351809b8c543"} Jun 08 22:48:20.728346 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.728309 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" event={"ID":"a9097ddc-23fa-46f4-9568-52ac7aa9abda","Type":"ContainerStarted","Data":"311ab740d7797d9e13911c78569bef8914f67dd0a36a674f002294a660be9792"} Jun 08 22:48:20.730486 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.730459 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xhk8k" event={"ID":"260a4693-1d70-4eea-ac68-9bf0ed2c87d9","Type":"ContainerStarted","Data":"c5d0eca2effb9ad3c2ab5af5b334a374d2a56ecd52b99eb8d8e083a50ef0afb8"} Jun 08 22:48:20.738033 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.737980 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-54swc" podStartSLOduration=12.538342145 podStartE2EDuration="22.737960405s" totalImagesPullingTime="10.19961826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:20.737248506 +0000 UTC m=+22.545860361" watchObservedRunningTime="2026-06-08 22:48:20.737960405 +0000 UTC m=+22.546572259" Jun 08 22:48:20.738165 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.738095 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c45cn" podStartSLOduration=4.121035945 podStartE2EDuration="22.738087404s" totalImagesPullingTime="18.617051459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:20.721338322 +0000 UTC m=+22.529950175" watchObservedRunningTime="2026-06-08 22:48:20.738087404 +0000 UTC m=+22.546699259" Jun 08 22:48:20.767653 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.767605 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kcmw6" podStartSLOduration=4.15378391 podStartE2EDuration="22.767591136s" totalImagesPullingTime="18.613807226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:20.767304284 +0000 UTC m=+22.575916141" watchObservedRunningTime="2026-06-08 22:48:20.767591136 +0000 UTC m=+22.576202989" Jun 08 22:48:20.779202 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.779157 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gpjcx" podStartSLOduration=4.154794292 podStartE2EDuration="22.779141706s" totalImagesPullingTime="18.624347414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:20.779029508 +0000 UTC m=+22.587641366" watchObservedRunningTime="2026-06-08 22:48:20.779141706 +0000 UTC m=+22.587753560" Jun 08 22:48:20.793104 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.792991 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xhk8k" podStartSLOduration=4.169318868 podStartE2EDuration="22.792975594s" totalImagesPullingTime="18.623656726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:20.792834959 +0000 UTC m=+22.601446813" watchObservedRunningTime="2026-06-08 22:48:20.792975594 +0000 UTC m=+22.601587448" Jun 08 22:48:20.932484 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:20.932450 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:21.013296 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.013269 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:21.113803 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.113725 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:21.327138 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.327111 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 08 22:48:21.612515 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.612414 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-08T22:48:21.327134534Z","UUID":"d35ad981-6cf2-4346-a192-ae0b0c0acb53","Handler":null,"Name":"","Endpoint":""} Jun 08 22:48:21.613879 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.613860 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 08 22:48:21.613990 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.613896 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 08 22:48:21.735071 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.735038 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5hnxw" event={"ID":"0446f325-1d19-4795-bec4-92298eaea1bf","Type":"ContainerStarted","Data":"012e245be389b488186bda3c7344c5a9c00ad2480ca5465fd1dde8947a6a578c"} Jun 08 22:48:21.737347 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.737295 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" event={"ID":"8bef3bb3-12d5-49dd-b50f-11f885052859","Type":"ContainerStarted","Data":"5cdbf0aa213d494d1fbda4f7e3681ca21a10d18321ff340448d4853f6ef336a6"} Jun 08 22:48:21.738294 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.738272 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:21.798845 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.798810 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:21.845201 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.845152 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:21.986563 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:21.986523 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:22.086513 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:22.086482 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:22.229522 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:22.229489 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:22.615919 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:22.615883 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:22.616110 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:22.615887 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:22.616110 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:22.615992 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:22.616110 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:22.616058 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:22.616110 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:22.615887 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:22.616327 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:22.616128 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:22.741836 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:22.741647 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" event={"ID":"8bef3bb3-12d5-49dd-b50f-11f885052859","Type":"ContainerStarted","Data":"fffec0506b4fdd4bbba014ff1d9feec9c0543f3023468719a97428cc1b88cb19"} Jun 08 22:48:22.766864 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:22.766806 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vp4n8" podStartSLOduration=4.099678715 podStartE2EDuration="24.766787132s" totalImagesPullingTime="20.667108417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:22.766709729 +0000 UTC m=+24.575321583" watchObservedRunningTime="2026-06-08 22:48:22.766787132 +0000 UTC m=+24.575398986" Jun 08 22:48:22.767552 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:22.767517 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5hnxw" podStartSLOduration=6.153152952 podStartE2EDuration="24.767501461s" totalImagesPullingTime="18.614348509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:21.746773454 +0000 UTC m=+23.555385308" watchObservedRunningTime="2026-06-08 22:48:22.767501461 +0000 UTC m=+24.576113318" Jun 08 22:48:23.389171 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.389117 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:23.489336 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.489302 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:23.535565 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.535531 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:23.672164 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.672126 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:23.672739 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.672718 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:23.741993 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.741948 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:23.747067 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.747047 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 22:48:23.747569 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.747535 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerStarted","Data":"e252733718a7ce9af5799d9661288302de1e9702e95573f46edd5967d0b4f173"} Jun 08 22:48:23.747666 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.747622 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:23.748229 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.748211 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-54swc" Jun 08 22:48:23.840835 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:23.840802 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:24.615635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:24.615602 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:24.615848 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:24.615663 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:24.615848 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:24.615602 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:24.615848 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:24.615781 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:24.615992 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:24.615953 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:24.616073 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:24.616052 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:25.753335 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:25.753130 2567 generic.go:356] "Generic (PLEG): container finished" podID="d30efbda-430f-4750-99ba-2da9000234f3" containerID="f60b7cb960c19b42083caf1f86e715e9f30f4345e4a076f8686385bfda05073b" exitCode=0 Jun 08 22:48:25.754147 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:25.753174 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-djtmb" event={"ID":"d30efbda-430f-4750-99ba-2da9000234f3","Type":"ContainerDied","Data":"f60b7cb960c19b42083caf1f86e715e9f30f4345e4a076f8686385bfda05073b"} Jun 08 22:48:25.757047 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:25.757028 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 22:48:25.757463 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:25.757426 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerStarted","Data":"2526d47045d0c3667c10a880ddfdade21430f03fc4a1744fba3bae8522fe7f41"} Jun 08 22:48:25.757911 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:25.757864 2567 scope.go:117] "RemoveContainer" containerID="ac20e063daf05fca65cd2fd6c0b2356701ea2b15ee3f6436e5bf34af6e98a3f3" Jun 08 22:48:25.990953 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:25.990923 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:26.615314 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.615278 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:26.615499 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.615278 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:26.615499 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:26.615407 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:26.615499 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.615429 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:26.615675 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:26.615532 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:26.615675 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:26.615633 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:26.764528 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.764505 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 22:48:26.764897 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.764851 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" event={"ID":"1b1f88ae-6de8-4d13-9f67-aa38624ff076","Type":"ContainerStarted","Data":"b394701d5afaa91d0ede08b4969972cd0e22f20bfbd476bc0e05511898a4db36"} Jun 08 22:48:26.765143 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.765118 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:26.765259 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.765202 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:26.781684 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.781651 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:26.781839 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.781802 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:26.791614 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.791570 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" podStartSLOduration=10.094394327 podStartE2EDuration="28.791554378s" totalImagesPullingTime="18.697160051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:26.790325109 +0000 UTC m=+28.598936962" watchObservedRunningTime="2026-06-08 22:48:26.791554378 +0000 UTC m=+28.600166233" Jun 08 22:48:26.944147 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:26.944102 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:27.033954 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:27.033915 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:27.227456 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:27.227421 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xs9jd"] Jun 08 22:48:27.227644 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:27.227552 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:27.227708 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:27.227658 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:27.233352 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:27.233324 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rds9j"] Jun 08 22:48:27.233725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:27.233706 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:27.233885 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:27.233861 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:27.234319 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:27.234265 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xwkrs"] Jun 08 22:48:27.234436 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:27.234357 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:27.234485 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:27.234448 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:27.769987 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:27.769739 2567 generic.go:356] "Generic (PLEG): container finished" podID="d30efbda-430f-4750-99ba-2da9000234f3" containerID="e5dfdb7dee32c495eac21200e02deb7b025dcb2233b82bd90947efd43cb76fba" exitCode=0 Jun 08 22:48:27.769987 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:27.769823 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-djtmb" event={"ID":"d30efbda-430f-4750-99ba-2da9000234f3","Type":"ContainerDied","Data":"e5dfdb7dee32c495eac21200e02deb7b025dcb2233b82bd90947efd43cb76fba"} Jun 08 22:48:28.616824 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:28.616794 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:28.616967 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:28.616847 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:28.617024 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:28.616998 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:28.617102 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:28.617078 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:28.617233 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:28.617161 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:28.617233 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:28.616990 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:28.774261 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:28.774226 2567 generic.go:356] "Generic (PLEG): container finished" podID="d30efbda-430f-4750-99ba-2da9000234f3" containerID="492493d05c56f2fd00a0cae65f60dc8a99eb8c64ffd4655a027f1438ef633b6b" exitCode=0 Jun 08 22:48:28.774788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:28.774310 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-djtmb" event={"ID":"d30efbda-430f-4750-99ba-2da9000234f3","Type":"ContainerDied","Data":"492493d05c56f2fd00a0cae65f60dc8a99eb8c64ffd4655a027f1438ef633b6b"} Jun 08 22:48:30.615475 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:30.615444 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:30.616183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:30.615444 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:30.616183 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:30.615607 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:30.616183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:30.615657 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:30.616183 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:30.615778 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:30.616183 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:30.615875 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:32.315458 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:32.315427 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:32.315982 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:32.315490 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:32.315982 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.315559 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:32.315982 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.315605 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:32.315982 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.315624 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs podName:fbb0f393-3467-4695-8c0f-d31e1f735f9c nodeName:}" failed. No retries permitted until 2026-06-08 22:49:04.315608921 +0000 UTC m=+66.124220757 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs") pod "network-metrics-daemon-rds9j" (UID: "fbb0f393-3467-4695-8c0f-d31e1f735f9c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:32.315982 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.315651 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret podName:c44e313d-1c49-4766-9a6b-daa3e68b9c2e nodeName:}" failed. No retries permitted until 2026-06-08 22:49:04.315640371 +0000 UTC m=+66.124252206 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret") pod "global-pull-secret-syncer-xs9jd" (UID: "c44e313d-1c49-4766-9a6b-daa3e68b9c2e") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:32.416475 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:32.416441 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:32.416643 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.416606 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:32.416643 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.416630 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:32.416643 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.416646 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6dlkj for pod openshift-network-diagnostics/network-check-target-xwkrs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:32.416770 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.416711 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj podName:49b1add5-a152-4d44-81f9-a7b7e4d8d12a nodeName:}" failed. No retries permitted until 2026-06-08 22:49:04.41669188 +0000 UTC m=+66.225303713 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dlkj" (UniqueName: "kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj") pod "network-check-target-xwkrs" (UID: "49b1add5-a152-4d44-81f9-a7b7e4d8d12a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:32.615435 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:32.615323 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:32.615435 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:32.615323 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:32.615655 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.615474 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xwkrs" podUID="49b1add5-a152-4d44-81f9-a7b7e4d8d12a" Jun 08 22:48:32.615655 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:32.615491 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:32.615655 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.615629 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:48:32.615821 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:32.615740 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9jd" podUID="c44e313d-1c49-4766-9a6b-daa3e68b9c2e" Jun 08 22:48:32.786998 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:32.786967 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-djtmb" event={"ID":"d30efbda-430f-4750-99ba-2da9000234f3","Type":"ContainerStarted","Data":"dd6df8a79999f0325b4340616601dc1ed77645e93bd50d4210dfbdfe8c5ec0ef"} Jun 08 22:48:33.027192 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.027158 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-248.ec2.internal" event="NodeReady" Jun 08 22:48:33.027361 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.027282 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 08 22:48:33.066426 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.066374 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9dmt5"] Jun 08 22:48:33.079891 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.079865 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z79nn"] Jun 08 22:48:33.080054 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.080003 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.081735 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.081701 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 08 22:48:33.081735 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.081730 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jnlbh\"" Jun 08 22:48:33.081927 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.081705 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 08 22:48:33.092007 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.091987 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9dmt5"] Jun 08 22:48:33.092110 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.092023 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z79nn"] Jun 08 22:48:33.092110 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.092096 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z79nn" Jun 08 22:48:33.093736 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.093711 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 08 22:48:33.093841 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.093804 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 08 22:48:33.093841 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.093830 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 08 22:48:33.093955 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.093881 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9wnvx\"" Jun 08 22:48:33.165135 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.165102 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:33.222182 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.222140 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1934d5fb-ba1e-48ec-86ee-ae2e12707285-tmp-dir\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.222306 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.222191 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l9f2\" (UniqueName: \"kubernetes.io/projected/3fa35cc0-fc3e-467e-b9a8-16da413b82ed-kube-api-access-4l9f2\") pod \"ingress-canary-z79nn\" (UID: \"3fa35cc0-fc3e-467e-b9a8-16da413b82ed\") " pod="openshift-ingress-canary/ingress-canary-z79nn" Jun 08 22:48:33.222306 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.222208 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8tjq\" (UniqueName: \"kubernetes.io/projected/1934d5fb-ba1e-48ec-86ee-ae2e12707285-kube-api-access-s8tjq\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.222411 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.222301 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1934d5fb-ba1e-48ec-86ee-ae2e12707285-config-volume\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.222411 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.222329 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3fa35cc0-fc3e-467e-b9a8-16da413b82ed-cert\") pod \"ingress-canary-z79nn\" (UID: \"3fa35cc0-fc3e-467e-b9a8-16da413b82ed\") " pod="openshift-ingress-canary/ingress-canary-z79nn" Jun 08 22:48:33.222411 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.222346 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.323449 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.323351 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1934d5fb-ba1e-48ec-86ee-ae2e12707285-tmp-dir\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.323449 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.323388 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l9f2\" (UniqueName: \"kubernetes.io/projected/3fa35cc0-fc3e-467e-b9a8-16da413b82ed-kube-api-access-4l9f2\") pod \"ingress-canary-z79nn\" (UID: \"3fa35cc0-fc3e-467e-b9a8-16da413b82ed\") " pod="openshift-ingress-canary/ingress-canary-z79nn" Jun 08 22:48:33.323913 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.323528 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8tjq\" (UniqueName: \"kubernetes.io/projected/1934d5fb-ba1e-48ec-86ee-ae2e12707285-kube-api-access-s8tjq\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.323913 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.323610 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1934d5fb-ba1e-48ec-86ee-ae2e12707285-config-volume\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.323913 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.323716 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1934d5fb-ba1e-48ec-86ee-ae2e12707285-tmp-dir\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.323913 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.323753 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3fa35cc0-fc3e-467e-b9a8-16da413b82ed-cert\") pod \"ingress-canary-z79nn\" (UID: \"3fa35cc0-fc3e-467e-b9a8-16da413b82ed\") " pod="openshift-ingress-canary/ingress-canary-z79nn" Jun 08 22:48:33.323913 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.323795 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.323913 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:33.323912 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:33.324154 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:33.323958 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls podName:1934d5fb-ba1e-48ec-86ee-ae2e12707285 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:33.823945159 +0000 UTC m=+35.632556991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls") pod "dns-default-9dmt5" (UID: "1934d5fb-ba1e-48ec-86ee-ae2e12707285") : secret "dns-default-metrics-tls" not found Jun 08 22:48:33.324154 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.324096 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1934d5fb-ba1e-48ec-86ee-ae2e12707285-config-volume\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.327759 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.327740 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3fa35cc0-fc3e-467e-b9a8-16da413b82ed-cert\") pod \"ingress-canary-z79nn\" (UID: \"3fa35cc0-fc3e-467e-b9a8-16da413b82ed\") " pod="openshift-ingress-canary/ingress-canary-z79nn" Jun 08 22:48:33.333519 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.333490 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8tjq\" (UniqueName: \"kubernetes.io/projected/1934d5fb-ba1e-48ec-86ee-ae2e12707285-kube-api-access-s8tjq\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.333615 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.333521 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l9f2\" (UniqueName: \"kubernetes.io/projected/3fa35cc0-fc3e-467e-b9a8-16da413b82ed-kube-api-access-4l9f2\") pod \"ingress-canary-z79nn\" (UID: \"3fa35cc0-fc3e-467e-b9a8-16da413b82ed\") " pod="openshift-ingress-canary/ingress-canary-z79nn" Jun 08 22:48:33.400616 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.400571 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z79nn" Jun 08 22:48:33.599432 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.599116 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z79nn"] Jun 08 22:48:33.602355 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.602308 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:33.603089 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:33.603064 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa35cc0_fc3e_467e_b9a8_16da413b82ed.slice/crio-2772e194ded664b1a8a9247b8da003e1ce431c5345c2eb05401046f39866f370 WatchSource:0}: Error finding container 2772e194ded664b1a8a9247b8da003e1ce431c5345c2eb05401046f39866f370: Status 404 returned error can't find the container with id 2772e194ded664b1a8a9247b8da003e1ce431c5345c2eb05401046f39866f370 Jun 08 22:48:33.697859 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.697830 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:33.790946 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.790906 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z79nn" event={"ID":"3fa35cc0-fc3e-467e-b9a8-16da413b82ed","Type":"ContainerStarted","Data":"2772e194ded664b1a8a9247b8da003e1ce431c5345c2eb05401046f39866f370"} Jun 08 22:48:33.793301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.793278 2567 generic.go:356] "Generic (PLEG): container finished" podID="d30efbda-430f-4750-99ba-2da9000234f3" containerID="dd6df8a79999f0325b4340616601dc1ed77645e93bd50d4210dfbdfe8c5ec0ef" exitCode=0 Jun 08 22:48:33.793424 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.793315 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-djtmb" event={"ID":"d30efbda-430f-4750-99ba-2da9000234f3","Type":"ContainerDied","Data":"dd6df8a79999f0325b4340616601dc1ed77645e93bd50d4210dfbdfe8c5ec0ef"} Jun 08 22:48:33.828048 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:33.827749 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:33.828048 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:33.827957 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:33.828048 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:33.828020 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls podName:1934d5fb-ba1e-48ec-86ee-ae2e12707285 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:34.828001267 +0000 UTC m=+36.636613116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls") pod "dns-default-9dmt5" (UID: "1934d5fb-ba1e-48ec-86ee-ae2e12707285") : secret "dns-default-metrics-tls" not found Jun 08 22:48:34.615795 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.615761 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:48:34.616321 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.615762 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:48:34.616321 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.615968 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:48:34.619252 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.619225 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 22:48:34.619620 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.619490 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 22:48:34.619620 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.619505 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 22:48:34.619620 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.619529 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k2dq5\"" Jun 08 22:48:34.619620 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.619557 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 22:48:34.619620 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.619563 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wjrnx\"" Jun 08 22:48:34.798519 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.798482 2567 generic.go:356] "Generic (PLEG): container finished" podID="d30efbda-430f-4750-99ba-2da9000234f3" containerID="d0974bfc624b9bc00dfb609418658ec5a1bc9086e69d168fc830a073ec8ee20c" exitCode=0 Jun 08 22:48:34.798675 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.798530 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-djtmb" event={"ID":"d30efbda-430f-4750-99ba-2da9000234f3","Type":"ContainerDied","Data":"d0974bfc624b9bc00dfb609418658ec5a1bc9086e69d168fc830a073ec8ee20c"} Jun 08 22:48:34.835798 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:34.835765 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:34.835938 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:34.835919 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:34.836009 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:34.835994 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls podName:1934d5fb-ba1e-48ec-86ee-ae2e12707285 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:36.835975664 +0000 UTC m=+38.644587508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls") pod "dns-default-9dmt5" (UID: "1934d5fb-ba1e-48ec-86ee-ae2e12707285") : secret "dns-default-metrics-tls" not found Jun 08 22:48:35.803857 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:35.803558 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-djtmb" event={"ID":"d30efbda-430f-4750-99ba-2da9000234f3","Type":"ContainerStarted","Data":"64fa148952bef54e7d0a198bfbd6d571bd996cc0276f23228b29bf747c678e7e"} Jun 08 22:48:35.804984 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:35.804964 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z79nn" event={"ID":"3fa35cc0-fc3e-467e-b9a8-16da413b82ed","Type":"ContainerStarted","Data":"f550282a1f02615a872934ccd09e3e07ade323017581c037d82b15880af61df0"} Jun 08 22:48:35.827904 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:35.827853 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-djtmb" podStartSLOduration=8.732727381 podStartE2EDuration="37.827838037s" totalImagesPullingTime="29.095110656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:35.826659268 +0000 UTC m=+37.635271122" watchObservedRunningTime="2026-06-08 22:48:35.827838037 +0000 UTC m=+37.636449890" Jun 08 22:48:35.837411 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:35.837369 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:35.839765 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:35.839714 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z79nn" podStartSLOduration=0.920885153 podStartE2EDuration="2.839700127s" totalImagesPullingTime="1.918814974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:35.83889723 +0000 UTC m=+37.647509097" watchObservedRunningTime="2026-06-08 22:48:35.839700127 +0000 UTC m=+37.648311958" Jun 08 22:48:35.886003 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:35.885973 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:36.856477 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:36.856437 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:36.857042 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:36.856597 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:36.857042 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:36.856661 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls podName:1934d5fb-ba1e-48ec-86ee-ae2e12707285 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:40.856646046 +0000 UTC m=+42.665257878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls") pod "dns-default-9dmt5" (UID: "1934d5fb-ba1e-48ec-86ee-ae2e12707285") : secret "dns-default-metrics-tls" not found Jun 08 22:48:37.124968 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:37.124887 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:37.213093 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:37.213049 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:37.268864 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:37.268824 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:37.565700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:37.565664 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:39.165236 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:39.165210 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:39.267554 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:39.267525 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:40.231488 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:40.231447 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:40.883061 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:40.883014 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:40.883251 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:40.883195 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:40.883296 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:40.883267 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls podName:1934d5fb-ba1e-48ec-86ee-ae2e12707285 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:48.88325143 +0000 UTC m=+50.691863262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls") pod "dns-default-9dmt5" (UID: "1934d5fb-ba1e-48ec-86ee-ae2e12707285") : secret "dns-default-metrics-tls" not found Jun 08 22:48:48.941301 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:48.941263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:48:48.941794 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:48.941381 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:48.941794 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:48:48.941464 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls podName:1934d5fb-ba1e-48ec-86ee-ae2e12707285 nodeName:}" failed. No retries permitted until 2026-06-08 22:49:04.94144534 +0000 UTC m=+66.750057175 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls") pod "dns-default-9dmt5" (UID: "1934d5fb-ba1e-48ec-86ee-ae2e12707285") : secret "dns-default-metrics-tls" not found Jun 08 22:48:56.269137 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.269099 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb"] Jun 08 22:48:56.317948 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.317919 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb"] Jun 08 22:48:56.317948 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.317949 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2"] Jun 08 22:48:56.318190 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.318060 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" Jun 08 22:48:56.320252 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.320227 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 08 22:48:56.320421 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.320267 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pjxcs\"" Jun 08 22:48:56.320421 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.320264 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 08 22:48:56.320421 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.320324 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 08 22:48:56.320647 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.320629 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 08 22:48:56.331243 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.331220 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2"] Jun 08 22:48:56.331342 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.331334 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.333095 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.333072 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 08 22:48:56.333214 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.333072 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 08 22:48:56.333214 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.333073 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 08 22:48:56.333214 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.333115 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 08 22:48:56.393437 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.393384 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.393641 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.393498 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t597c\" (UniqueName: \"kubernetes.io/projected/f44ca8db-3dc8-484c-9451-132c695caddd-kube-api-access-t597c\") pod \"managed-serviceaccount-addon-agent-84b55c8cb-kphpb\" (UID: \"f44ca8db-3dc8-484c-9451-132c695caddd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" Jun 08 22:48:56.393641 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.393556 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e1caf96-73b3-4d34-a4da-abddc07d29c9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.393641 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.393602 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-hub\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.393641 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.393625 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nssv2\" (UniqueName: \"kubernetes.io/projected/7e1caf96-73b3-4d34-a4da-abddc07d29c9-kube-api-access-nssv2\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.393808 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.393656 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.393808 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.393696 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f44ca8db-3dc8-484c-9451-132c695caddd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b55c8cb-kphpb\" (UID: \"f44ca8db-3dc8-484c-9451-132c695caddd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" Jun 08 22:48:56.393808 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.393718 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-ca\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.494906 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.494865 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e1caf96-73b3-4d34-a4da-abddc07d29c9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.495081 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.494927 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-hub\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.495081 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.495049 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nssv2\" (UniqueName: \"kubernetes.io/projected/7e1caf96-73b3-4d34-a4da-abddc07d29c9-kube-api-access-nssv2\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.495198 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.495099 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.495198 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.495134 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f44ca8db-3dc8-484c-9451-132c695caddd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b55c8cb-kphpb\" (UID: \"f44ca8db-3dc8-484c-9451-132c695caddd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" Jun 08 22:48:56.495198 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.495157 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-ca\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.495335 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.495217 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.495335 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.495269 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t597c\" (UniqueName: \"kubernetes.io/projected/f44ca8db-3dc8-484c-9451-132c695caddd-kube-api-access-t597c\") pod \"managed-serviceaccount-addon-agent-84b55c8cb-kphpb\" (UID: \"f44ca8db-3dc8-484c-9451-132c695caddd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" Jun 08 22:48:56.495892 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.495864 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e1caf96-73b3-4d34-a4da-abddc07d29c9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.499141 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.499111 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-hub\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.499258 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.499189 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-ca\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.499258 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.499236 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.499344 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.499279 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e1caf96-73b3-4d34-a4da-abddc07d29c9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.499423 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.499381 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f44ca8db-3dc8-484c-9451-132c695caddd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b55c8cb-kphpb\" (UID: \"f44ca8db-3dc8-484c-9451-132c695caddd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" Jun 08 22:48:56.501642 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.501622 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t597c\" (UniqueName: \"kubernetes.io/projected/f44ca8db-3dc8-484c-9451-132c695caddd-kube-api-access-t597c\") pod \"managed-serviceaccount-addon-agent-84b55c8cb-kphpb\" (UID: \"f44ca8db-3dc8-484c-9451-132c695caddd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" Jun 08 22:48:56.501730 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.501689 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nssv2\" (UniqueName: \"kubernetes.io/projected/7e1caf96-73b3-4d34-a4da-abddc07d29c9-kube-api-access-nssv2\") pod \"cluster-proxy-proxy-agent-56f948979c-c5bs2\" (UID: \"7e1caf96-73b3-4d34-a4da-abddc07d29c9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.636925 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.636842 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" Jun 08 22:48:56.643030 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.643004 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:48:56.803451 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.803368 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb"] Jun 08 22:48:56.807309 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:56.807284 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf44ca8db_3dc8_484c_9451_132c695caddd.slice/crio-61333e2162e82e424d096d0a0e4b18cd128e8ffeccbbedf4c79e3838849879c1 WatchSource:0}: Error finding container 61333e2162e82e424d096d0a0e4b18cd128e8ffeccbbedf4c79e3838849879c1: Status 404 returned error can't find the container with id 61333e2162e82e424d096d0a0e4b18cd128e8ffeccbbedf4c79e3838849879c1 Jun 08 22:48:56.819295 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.819263 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2"] Jun 08 22:48:56.823455 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:48:56.823431 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e1caf96_73b3_4d34_a4da_abddc07d29c9.slice/crio-1e0388510a4a7d8d98cfd5a6491cc4be90aef63f20e49928517b7ec2164a3f17 WatchSource:0}: Error finding container 1e0388510a4a7d8d98cfd5a6491cc4be90aef63f20e49928517b7ec2164a3f17: Status 404 returned error can't find the container with id 1e0388510a4a7d8d98cfd5a6491cc4be90aef63f20e49928517b7ec2164a3f17 Jun 08 22:48:56.853046 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.853013 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" event={"ID":"f44ca8db-3dc8-484c-9451-132c695caddd","Type":"ContainerStarted","Data":"61333e2162e82e424d096d0a0e4b18cd128e8ffeccbbedf4c79e3838849879c1"} Jun 08 22:48:56.854031 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:56.854008 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" event={"ID":"7e1caf96-73b3-4d34-a4da-abddc07d29c9","Type":"ContainerStarted","Data":"1e0388510a4a7d8d98cfd5a6491cc4be90aef63f20e49928517b7ec2164a3f17"} Jun 08 22:48:58.788518 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:58.788174 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9hb5j" Jun 08 22:48:58.934322 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:58.934286 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:58.983511 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:58.983468 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:59.045433 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:59.045315 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:59.826277 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:59.826245 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:48:59.876128 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:48:59.876094 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:49:00.345655 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:00.345620 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:49:01.869076 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:01.869039 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" event={"ID":"7e1caf96-73b3-4d34-a4da-abddc07d29c9","Type":"ContainerStarted","Data":"70412fbac0a95d16704368dcf284c314587d906d56497fd2ff75bba1a8b7dc9c"} Jun 08 22:49:01.869546 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:01.869089 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" event={"ID":"7e1caf96-73b3-4d34-a4da-abddc07d29c9","Type":"ContainerStarted","Data":"1a3d1d214e7ceb2ba5266069c7f9e3c0b75c7718e78c5080362b8f15ee6c15a6"} Jun 08 22:49:01.869546 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:01.869108 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" event={"ID":"7e1caf96-73b3-4d34-a4da-abddc07d29c9","Type":"ContainerStarted","Data":"0506e85401f05a10c850da5986ce252aaa78ac0ea6a8cc4052a5d35ad1502e17"} Jun 08 22:49:01.870359 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:01.870329 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" event={"ID":"f44ca8db-3dc8-484c-9451-132c695caddd","Type":"ContainerStarted","Data":"95f6f40e70d8e4a7dfc424d1d8838ddc313a582dbe242f914a3ea0d676a66c02"} Jun 08 22:49:01.885009 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:01.884958 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" podStartSLOduration=1.271970579 podStartE2EDuration="5.884944609s" totalImagesPullingTime="4.61297403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:49:01.883962525 +0000 UTC m=+63.692574378" watchObservedRunningTime="2026-06-08 22:49:01.884944609 +0000 UTC m=+63.693556476" Jun 08 22:49:01.895548 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:01.895500 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" podStartSLOduration=1.283367848 podStartE2EDuration="5.895486454s" totalImagesPullingTime="4.612118606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:49:01.89495661 +0000 UTC m=+63.703568464" watchObservedRunningTime="2026-06-08 22:49:01.895486454 +0000 UTC m=+63.704098308" Jun 08 22:49:04.350890 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.350842 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:49:04.351352 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.350934 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:49:04.353195 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.353178 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 22:49:04.353295 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.353277 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 22:49:04.361629 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:49:04.361606 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 22:49:04.361715 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:49:04.361681 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs podName:fbb0f393-3467-4695-8c0f-d31e1f735f9c nodeName:}" failed. No retries permitted until 2026-06-08 22:50:08.361659473 +0000 UTC m=+130.170271327 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs") pod "network-metrics-daemon-rds9j" (UID: "fbb0f393-3467-4695-8c0f-d31e1f735f9c") : secret "metrics-daemon-secret" not found Jun 08 22:49:04.363680 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.363661 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c44e313d-1c49-4766-9a6b-daa3e68b9c2e-original-pull-secret\") pod \"global-pull-secret-syncer-xs9jd\" (UID: \"c44e313d-1c49-4766-9a6b-daa3e68b9c2e\") " pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:49:04.451737 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.451688 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:49:04.453894 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.453872 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 22:49:04.464572 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.464548 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 22:49:04.475111 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.475086 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dlkj\" (UniqueName: \"kubernetes.io/projected/49b1add5-a152-4d44-81f9-a7b7e4d8d12a-kube-api-access-6dlkj\") pod \"network-check-target-xwkrs\" (UID: \"49b1add5-a152-4d44-81f9-a7b7e4d8d12a\") " pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:49:04.636839 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.636760 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k2dq5\"" Jun 08 22:49:04.640193 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.640177 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9jd" Jun 08 22:49:04.645962 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.645939 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:49:04.765919 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.765886 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xs9jd"] Jun 08 22:49:04.768884 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:49:04.768856 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc44e313d_1c49_4766_9a6b_daa3e68b9c2e.slice/crio-1b9bdf9691f05551ff6d422514cbbe715de38149910151033290f7fb63eb6139 WatchSource:0}: Error finding container 1b9bdf9691f05551ff6d422514cbbe715de38149910151033290f7fb63eb6139: Status 404 returned error can't find the container with id 1b9bdf9691f05551ff6d422514cbbe715de38149910151033290f7fb63eb6139 Jun 08 22:49:04.781700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.781669 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xwkrs"] Jun 08 22:49:04.786311 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:49:04.786285 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b1add5_a152_4d44_81f9_a7b7e4d8d12a.slice/crio-fd9b586728b83571b48f90cc0850ec5ae574fb16351baf12db4f8db50bb14cb3 WatchSource:0}: Error finding container fd9b586728b83571b48f90cc0850ec5ae574fb16351baf12db4f8db50bb14cb3: Status 404 returned error can't find the container with id fd9b586728b83571b48f90cc0850ec5ae574fb16351baf12db4f8db50bb14cb3 Jun 08 22:49:04.881402 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.881354 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xwkrs" event={"ID":"49b1add5-a152-4d44-81f9-a7b7e4d8d12a","Type":"ContainerStarted","Data":"fd9b586728b83571b48f90cc0850ec5ae574fb16351baf12db4f8db50bb14cb3"} Jun 08 22:49:04.882427 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.882387 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xs9jd" event={"ID":"c44e313d-1c49-4766-9a6b-daa3e68b9c2e","Type":"ContainerStarted","Data":"1b9bdf9691f05551ff6d422514cbbe715de38149910151033290f7fb63eb6139"} Jun 08 22:49:04.955599 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:04.955564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:49:04.955744 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:49:04.955699 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:49:04.955801 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:49:04.955750 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls podName:1934d5fb-ba1e-48ec-86ee-ae2e12707285 nodeName:}" failed. No retries permitted until 2026-06-08 22:49:36.955736942 +0000 UTC m=+98.764348775 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls") pod "dns-default-9dmt5" (UID: "1934d5fb-ba1e-48ec-86ee-ae2e12707285") : secret "dns-default-metrics-tls" not found Jun 08 22:49:09.898947 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:09.898901 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xwkrs" event={"ID":"49b1add5-a152-4d44-81f9-a7b7e4d8d12a","Type":"ContainerStarted","Data":"3d771ffc5474acc85da90bb03dd53a58bd5740e5b813516302006e0c66c55592"} Jun 08 22:49:09.899366 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:09.899026 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:49:09.900420 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:09.900385 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xs9jd" event={"ID":"c44e313d-1c49-4766-9a6b-daa3e68b9c2e","Type":"ContainerStarted","Data":"cc6ae0d9aa6e1c82e08fd87d949a50444cb5da52c91262f1d943a2f5f9659c56"} Jun 08 22:49:09.911557 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:09.911509 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xwkrs" podStartSLOduration=67.622644964 podStartE2EDuration="1m11.911490994s" totalImagesPullingTime="4.28884603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:49:09.91116245 +0000 UTC m=+71.719774304" watchObservedRunningTime="2026-06-08 22:49:09.911490994 +0000 UTC m=+71.720102850" Jun 08 22:49:09.923214 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:09.923173 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xs9jd" podStartSLOduration=67.613833306 podStartE2EDuration="1m11.923157375s" totalImagesPullingTime="4.309324069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:49:09.922784152 +0000 UTC m=+71.731396009" watchObservedRunningTime="2026-06-08 22:49:09.923157375 +0000 UTC m=+71.731769229" Jun 08 22:49:36.981910 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:36.981876 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:49:36.982323 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:49:36.981987 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:49:36.982323 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:49:36.982046 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls podName:1934d5fb-ba1e-48ec-86ee-ae2e12707285 nodeName:}" failed. No retries permitted until 2026-06-08 22:50:40.982031543 +0000 UTC m=+162.790643375 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls") pod "dns-default-9dmt5" (UID: "1934d5fb-ba1e-48ec-86ee-ae2e12707285") : secret "dns-default-metrics-tls" not found Jun 08 22:49:40.905510 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:49:40.905477 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xwkrs" Jun 08 22:50:08.397627 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:50:08.397581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:50:08.398102 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:50:08.397740 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 22:50:08.398102 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:50:08.397806 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs podName:fbb0f393-3467-4695-8c0f-d31e1f735f9c nodeName:}" failed. No retries permitted until 2026-06-08 22:52:10.397790194 +0000 UTC m=+252.206402026 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs") pod "network-metrics-daemon-rds9j" (UID: "fbb0f393-3467-4695-8c0f-d31e1f735f9c") : secret "metrics-daemon-secret" not found Jun 08 22:50:36.089362 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:50:36.089307 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9dmt5" podUID="1934d5fb-ba1e-48ec-86ee-ae2e12707285" Jun 08 22:50:36.143818 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:50:36.143790 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9dmt5" Jun 08 22:50:37.627281 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:50:37.627224 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rds9j" podUID="fbb0f393-3467-4695-8c0f-d31e1f735f9c" Jun 08 22:50:41.023631 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:50:41.023593 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:50:41.024110 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:50:41.023713 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:50:41.024110 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:50:41.023762 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls podName:1934d5fb-ba1e-48ec-86ee-ae2e12707285 nodeName:}" failed. No retries permitted until 2026-06-08 22:52:43.023748717 +0000 UTC m=+284.832360550 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls") pod "dns-default-9dmt5" (UID: "1934d5fb-ba1e-48ec-86ee-ae2e12707285") : secret "dns-default-metrics-tls" not found Jun 08 22:50:48.617053 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:50:48.617020 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:50:50.362661 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:50:50.362621 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:02.222623 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:02.222587 2567 generic.go:356] "Generic (PLEG): container finished" podID="f44ca8db-3dc8-484c-9451-132c695caddd" containerID="95f6f40e70d8e4a7dfc424d1d8838ddc313a582dbe242f914a3ea0d676a66c02" exitCode=255 Jun 08 22:51:02.222999 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:02.222660 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" event={"ID":"f44ca8db-3dc8-484c-9451-132c695caddd","Type":"ContainerDied","Data":"95f6f40e70d8e4a7dfc424d1d8838ddc313a582dbe242f914a3ea0d676a66c02"} Jun 08 22:51:02.222999 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:02.222967 2567 scope.go:117] "RemoveContainer" containerID="95f6f40e70d8e4a7dfc424d1d8838ddc313a582dbe242f914a3ea0d676a66c02" Jun 08 22:51:03.227460 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:03.227425 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b55c8cb-kphpb" event={"ID":"f44ca8db-3dc8-484c-9451-132c695caddd","Type":"ContainerStarted","Data":"66449f2c80f184a28fcbdfb693eb578c7ab675baad02fc823b41075fc800ecfc"} Jun 08 22:51:12.023904 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:12.023871 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:13.188972 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:13.188938 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:13.677931 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:13.677899 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:13.723770 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:13.723736 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:16.135678 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:16.135645 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:16.232761 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:16.232730 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:16.332807 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:16.332772 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:16.433024 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:16.432993 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:16.693289 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:16.693206 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:16.736791 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:16.736763 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:16.961277 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:16.961192 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:17.062187 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:17.062163 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:17.704797 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:17.704769 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:17.805116 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:17.805088 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:18.595850 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:18.595820 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:18.645671 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:18.645639 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:18.960479 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:18.960453 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gpjcx_f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e/dns-node-resolver/0.log" Jun 08 22:51:19.960529 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:19.960501 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c45cn_b8c10f1b-1d19-4d08-b676-8434b760c799/node-ca/0.log" Jun 08 22:51:21.161495 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:21.161468 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z79nn_3fa35cc0-fc3e-467e-b9a8-16da413b82ed/serve-healthcheck-canary/0.log" Jun 08 22:51:21.706437 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:21.706409 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:21.835919 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:21.835885 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:21.942665 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:21.942635 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:22.037300 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:22.037228 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:22.209407 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:22.209371 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:22.261374 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:22.261345 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:22.475635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:22.475601 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:23.200296 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:23.200266 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:23.793989 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:23.793959 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:23.997259 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:23.997228 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:24.399603 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:24.399566 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:24.603430 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:24.603383 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:25.007372 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.007344 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:25.516367 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.516338 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:25.522700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.522677 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-ccd755cdc-76dgr"] Jun 08 22:51:25.525463 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.525447 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.527334 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.527311 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 08 22:51:25.527557 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.527542 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 08 22:51:25.527625 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.527542 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 08 22:51:25.527667 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.527622 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bpm8k\"" Jun 08 22:51:25.533248 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.533230 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 08 22:51:25.540598 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.538431 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ccd755cdc-76dgr"] Jun 08 22:51:25.615716 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.615684 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:25.646382 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.646350 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-bound-sa-token\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.646539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.646410 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-installation-pull-secrets\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.646539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.646476 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-registry-tls\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.646539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.646501 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ss55\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-kube-api-access-7ss55\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.646539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.646521 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-image-registry-private-configuration\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.646539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.646540 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06432f82-5309-4416-add3-3b0b77e7397e-ca-trust-extracted\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.646718 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.646555 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-registry-certificates\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.646718 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.646644 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-trusted-ca\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.717141 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.717110 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:25.747055 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747025 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-trusted-ca\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.747169 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747086 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-bound-sa-token\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.747169 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747111 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-installation-pull-secrets\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.747169 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747133 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-registry-tls\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.747169 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747149 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ss55\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-kube-api-access-7ss55\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.747169 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747166 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-image-registry-private-configuration\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.747463 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747182 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06432f82-5309-4416-add3-3b0b77e7397e-ca-trust-extracted\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.747463 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747256 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-registry-certificates\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.747637 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747617 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06432f82-5309-4416-add3-3b0b77e7397e-ca-trust-extracted\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.747888 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.747861 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-registry-certificates\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.748099 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.748074 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-trusted-ca\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.749671 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.749648 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-installation-pull-secrets\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.749753 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.749718 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-registry-tls\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.749811 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.749793 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-image-registry-private-configuration\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.753676 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.753652 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-bound-sa-token\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.753753 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.753723 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ss55\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-kube-api-access-7ss55\") pod \"image-registry-ccd755cdc-76dgr\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.835120 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.835030 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:25.959142 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:25.959104 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ccd755cdc-76dgr"] Jun 08 22:51:25.961912 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:51:25.961887 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06432f82_5309_4416_add3_3b0b77e7397e.slice/crio-10e3b6a59f2b73e9a994ceb9f227cb10f75d60c4d174d8c31cb796df82c251a9 WatchSource:0}: Error finding container 10e3b6a59f2b73e9a994ceb9f227cb10f75d60c4d174d8c31cb796df82c251a9: Status 404 returned error can't find the container with id 10e3b6a59f2b73e9a994ceb9f227cb10f75d60c4d174d8c31cb796df82c251a9 Jun 08 22:51:26.000120 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:26.000092 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:26.199193 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:26.199160 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:26.293237 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:26.293199 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" event={"ID":"06432f82-5309-4416-add3-3b0b77e7397e","Type":"ContainerStarted","Data":"7d0a1edabafd0116548d2db5150926aaf697a20725f3dbf98ba78c113f96ffc5"} Jun 08 22:51:26.293450 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:26.293257 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" event={"ID":"06432f82-5309-4416-add3-3b0b77e7397e","Type":"ContainerStarted","Data":"10e3b6a59f2b73e9a994ceb9f227cb10f75d60c4d174d8c31cb796df82c251a9"} Jun 08 22:51:26.293450 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:26.293327 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:51:26.309384 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:26.309332 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podStartSLOduration=1.3093163780000001 podStartE2EDuration="1.309316378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:51:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:26.308659715 +0000 UTC m=+208.117271582" watchObservedRunningTime="2026-06-08 22:51:26.309316378 +0000 UTC m=+208.117928233" Jun 08 22:51:26.796919 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:26.796890 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:27.000930 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:27.000899 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:27.599777 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:27.599748 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:27.997488 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:27.997454 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:28.112304 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:28.112256 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:28.211642 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:28.211610 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:28.600499 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:28.600467 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:29.199234 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.199199 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:29.299299 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.299267 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:29.312163 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.312136 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sv6sf"] Jun 08 22:51:29.316114 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.316093 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.318251 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.318221 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 08 22:51:29.321859 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.321740 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pcscd\"" Jun 08 22:51:29.321859 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.321754 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 08 22:51:29.322038 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.321913 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 08 22:51:29.322100 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.322082 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 08 22:51:29.323800 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.323776 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sv6sf"] Jun 08 22:51:29.476325 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.476231 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.476325 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.476286 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-crio-socket\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.476535 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.476345 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.476535 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.476504 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qphcw\" (UniqueName: \"kubernetes.io/projected/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-kube-api-access-qphcw\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.476600 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.476536 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-data-volume\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.577461 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.577423 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qphcw\" (UniqueName: \"kubernetes.io/projected/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-kube-api-access-qphcw\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.577461 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.577461 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-data-volume\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.577694 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.577484 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.577694 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.577514 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-crio-socket\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.577694 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.577592 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-crio-socket\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.577829 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.577696 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.577829 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:51:29.577821 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 08 22:51:29.577919 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:51:29.577890 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls podName:cb4c6bc3-bb3f-44d6-8af1-78866aa10258 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:30.077871381 +0000 UTC m=+211.886483216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sv6sf" (UID: "cb4c6bc3-bb3f-44d6-8af1-78866aa10258") : secret "insights-runtime-extractor-tls" not found Jun 08 22:51:29.577919 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.577821 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-data-volume\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.578133 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.578114 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.590211 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.590187 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qphcw\" (UniqueName: \"kubernetes.io/projected/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-kube-api-access-qphcw\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:29.604584 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:29.604556 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:30.001208 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:30.001177 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:30.081681 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:30.081647 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:30.081806 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:51:30.081790 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 08 22:51:30.081876 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:51:30.081867 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls podName:cb4c6bc3-bb3f-44d6-8af1-78866aa10258 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:31.081850738 +0000 UTC m=+212.890462570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sv6sf" (UID: "cb4c6bc3-bb3f-44d6-8af1-78866aa10258") : secret "insights-runtime-extractor-tls" not found Jun 08 22:51:30.402334 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:30.402254 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:30.597497 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:30.597470 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:31.089601 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:31.089567 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:31.090013 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:51:31.089725 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 08 22:51:31.090013 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:51:31.089788 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls podName:cb4c6bc3-bb3f-44d6-8af1-78866aa10258 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:33.089774312 +0000 UTC m=+214.898386150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sv6sf" (UID: "cb4c6bc3-bb3f-44d6-8af1-78866aa10258") : secret "insights-runtime-extractor-tls" not found Jun 08 22:51:31.197161 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:31.197129 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:31.600093 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:31.600063 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:31.800018 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:31.799980 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:32.201093 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:32.201061 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:32.396129 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:32.396093 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:33.106219 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:33.106184 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:33.108462 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:33.108433 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb4c6bc3-bb3f-44d6-8af1-78866aa10258-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sv6sf\" (UID: \"cb4c6bc3-bb3f-44d6-8af1-78866aa10258\") " pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:33.229940 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:33.229893 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sv6sf" Jun 08 22:51:33.356737 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:33.356671 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sv6sf"] Jun 08 22:51:33.360116 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:51:33.360087 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4c6bc3_bb3f_44d6_8af1_78866aa10258.slice/crio-0c9d7eab174f6b0f00e278e9c8dc0204c9561f1c261ff0f8b8c7176cdc2822de WatchSource:0}: Error finding container 0c9d7eab174f6b0f00e278e9c8dc0204c9561f1c261ff0f8b8c7176cdc2822de: Status 404 returned error can't find the container with id 0c9d7eab174f6b0f00e278e9c8dc0204c9561f1c261ff0f8b8c7176cdc2822de Jun 08 22:51:34.318651 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:34.318563 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sv6sf" event={"ID":"cb4c6bc3-bb3f-44d6-8af1-78866aa10258","Type":"ContainerStarted","Data":"44c7f93f13ffe38272db24e55065062fe0dc767d9511f993053a8adce25d3add"} Jun 08 22:51:34.318651 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:34.318626 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sv6sf" event={"ID":"cb4c6bc3-bb3f-44d6-8af1-78866aa10258","Type":"ContainerStarted","Data":"982cd858bdcdaa53a249ce928c0e1b87b42b61ca1a96c171fd67160e6ec7dca2"} Jun 08 22:51:34.318651 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:34.318653 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sv6sf" event={"ID":"cb4c6bc3-bb3f-44d6-8af1-78866aa10258","Type":"ContainerStarted","Data":"0c9d7eab174f6b0f00e278e9c8dc0204c9561f1c261ff0f8b8c7176cdc2822de"} Jun 08 22:51:35.379069 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:35.379035 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:35.478785 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:35.478756 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:35.524635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:35.524608 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:36.044519 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:36.044486 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:36.145430 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:36.145383 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:36.326715 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:36.326633 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sv6sf" event={"ID":"cb4c6bc3-bb3f-44d6-8af1-78866aa10258","Type":"ContainerStarted","Data":"7f99a245c8047b528d7b69ced1023206d8556d1fbbbf02cd189885a5cab28ca6"} Jun 08 22:51:36.342117 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:36.342070 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sv6sf" podStartSLOduration=5.307185057 podStartE2EDuration="7.342055571s" totalImagesPullingTime="2.034870514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:51:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:36.341011133 +0000 UTC m=+218.149622986" watchObservedRunningTime="2026-06-08 22:51:36.342055571 +0000 UTC m=+218.150667424" Jun 08 22:51:36.449073 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:36.449045 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:36.644116 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:36.644026 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" podUID="7e1caf96-73b3-4d34-a4da-abddc07d29c9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 08 22:51:36.673772 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:36.673739 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:36.725064 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:36.725031 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:37.055724 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:37.055692 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:37.105876 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:37.105837 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:37.261931 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:37.261884 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:37.458037 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:37.458006 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:38.867158 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:38.867118 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:41.932688 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:41.932662 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:45.840015 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:45.839983 2567 patch_prober.go:28] interesting pod/image-registry-ccd755cdc-76dgr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:51:45.840435 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:45.840033 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:51:46.607621 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:46.607589 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:46.644462 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:46.644427 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" podUID="7e1caf96-73b3-4d34-a4da-abddc07d29c9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 08 22:51:46.711282 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:46.711251 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:46.776905 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:46.776868 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:46.927365 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:46.927329 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:47.300774 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:47.300697 2567 patch_prober.go:28] interesting pod/image-registry-ccd755cdc-76dgr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:51:47.300774 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:47.300745 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:51:49.682723 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:49.682687 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:49.733131 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:49.733098 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:54.281636 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:54.281598 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:54.352629 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:54.352598 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:55.839061 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:55.839027 2567 patch_prober.go:28] interesting pod/image-registry-ccd755cdc-76dgr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:51:55.839446 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:55.839077 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:51:55.883607 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:55.883579 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:55.984350 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:55.984316 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:56.644222 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:56.644190 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" podUID="7e1caf96-73b3-4d34-a4da-abddc07d29c9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 08 22:51:56.644388 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:56.644255 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" Jun 08 22:51:56.644821 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:56.644792 2567 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"70412fbac0a95d16704368dcf284c314587d906d56497fd2ff75bba1a8b7dc9c"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 08 22:51:56.644858 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:56.644844 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" podUID="7e1caf96-73b3-4d34-a4da-abddc07d29c9" containerName="service-proxy" containerID="cri-o://70412fbac0a95d16704368dcf284c314587d906d56497fd2ff75bba1a8b7dc9c" gracePeriod=30 Jun 08 22:51:56.781105 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:56.781073 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:56.844438 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:56.844375 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:57.221237 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:57.221204 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:57.300684 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:57.300654 2567 patch_prober.go:28] interesting pod/image-registry-ccd755cdc-76dgr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:51:57.300813 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:57.300697 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:51:57.390009 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:57.389978 2567 generic.go:356] "Generic (PLEG): container finished" podID="7e1caf96-73b3-4d34-a4da-abddc07d29c9" containerID="70412fbac0a95d16704368dcf284c314587d906d56497fd2ff75bba1a8b7dc9c" exitCode=2 Jun 08 22:51:57.390162 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:57.390025 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" event={"ID":"7e1caf96-73b3-4d34-a4da-abddc07d29c9","Type":"ContainerDied","Data":"70412fbac0a95d16704368dcf284c314587d906d56497fd2ff75bba1a8b7dc9c"} Jun 08 22:51:57.390162 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:57.390061 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56f948979c-c5bs2" event={"ID":"7e1caf96-73b3-4d34-a4da-abddc07d29c9","Type":"ContainerStarted","Data":"e8c3a5d450e1dcc796b8f75d365738ddaf6cb6fdace9b64a2e6ac8e7932e2054"} Jun 08 22:51:59.259636 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:59.259601 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:59.408471 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:59.408440 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:59.546339 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:59.546248 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:59.648265 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:59.648233 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:59.702623 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:59.702596 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:59.880825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:59.880712 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:51:59.925810 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:51:59.925783 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:00.910656 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:00.910628 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:01.135365 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.135334 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:01.193617 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.193579 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sq5p4"] Jun 08 22:52:01.197060 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.197043 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.199086 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.199061 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 08 22:52:01.199184 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.199158 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 08 22:52:01.199184 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.199167 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ksq6q\"" Jun 08 22:52:01.199674 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.199644 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 08 22:52:01.199674 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.199669 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 08 22:52:01.199816 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.199652 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 08 22:52:01.199816 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.199709 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 08 22:52:01.218167 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.218141 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:01.287164 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.287131 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:01.311245 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.311215 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a03609a4-578a-4cca-80da-563aa1fb2aa7-metrics-client-ca\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.311388 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.311253 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-accelerators-collector-config\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.311388 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.311280 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a03609a4-578a-4cca-80da-563aa1fb2aa7-root\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.311388 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.311356 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-textfile\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.311581 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.311412 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-wtmp\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.311581 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.311442 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhz4r\" (UniqueName: \"kubernetes.io/projected/a03609a4-578a-4cca-80da-563aa1fb2aa7-kube-api-access-vhz4r\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.311581 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.311468 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-tls\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.311581 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.311515 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a03609a4-578a-4cca-80da-563aa1fb2aa7-sys\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.311581 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.311565 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.340935 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.340907 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:01.412270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412235 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a03609a4-578a-4cca-80da-563aa1fb2aa7-metrics-client-ca\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412272 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-accelerators-collector-config\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412526 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412306 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a03609a4-578a-4cca-80da-563aa1fb2aa7-root\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412526 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412358 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a03609a4-578a-4cca-80da-563aa1fb2aa7-root\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412526 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412408 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-textfile\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412526 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412448 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-wtmp\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412526 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhz4r\" (UniqueName: \"kubernetes.io/projected/a03609a4-578a-4cca-80da-563aa1fb2aa7-kube-api-access-vhz4r\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412526 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412501 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-tls\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412805 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412534 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a03609a4-578a-4cca-80da-563aa1fb2aa7-sys\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412805 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412579 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412805 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412610 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-wtmp\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.412805 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412672 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a03609a4-578a-4cca-80da-563aa1fb2aa7-sys\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.413008 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412847 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-textfile\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.413008 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.412976 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a03609a4-578a-4cca-80da-563aa1fb2aa7-metrics-client-ca\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.413143 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.413125 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-accelerators-collector-config\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.414946 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.414927 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-tls\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.415083 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.415067 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a03609a4-578a-4cca-80da-563aa1fb2aa7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.418953 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.418935 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhz4r\" (UniqueName: \"kubernetes.io/projected/a03609a4-578a-4cca-80da-563aa1fb2aa7-kube-api-access-vhz4r\") pod \"node-exporter-sq5p4\" (UID: \"a03609a4-578a-4cca-80da-563aa1fb2aa7\") " pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.427579 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.427553 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:01.506133 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.506045 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sq5p4" Jun 08 22:52:01.517045 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:52:01.517007 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03609a4_578a_4cca_80da_563aa1fb2aa7.slice/crio-2af221a44c51c98bf737ca3f985992ff82d72b85e53fb409215fcddb43afc17c WatchSource:0}: Error finding container 2af221a44c51c98bf737ca3f985992ff82d72b85e53fb409215fcddb43afc17c: Status 404 returned error can't find the container with id 2af221a44c51c98bf737ca3f985992ff82d72b85e53fb409215fcddb43afc17c Jun 08 22:52:01.528783 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.528757 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:01.729241 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:01.729204 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:02.144619 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:02.144574 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:02.209028 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:02.208986 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:02.406338 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:02.406262 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sq5p4" event={"ID":"a03609a4-578a-4cca-80da-563aa1fb2aa7","Type":"ContainerStarted","Data":"2af221a44c51c98bf737ca3f985992ff82d72b85e53fb409215fcddb43afc17c"} Jun 08 22:52:02.558266 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:02.558205 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:02.658090 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:02.658010 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:03.411021 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:03.410973 2567 generic.go:356] "Generic (PLEG): container finished" podID="a03609a4-578a-4cca-80da-563aa1fb2aa7" containerID="1baaaebaa3d406101fba17338b6c99418d237137b643704c090b2ad0b596bf88" exitCode=0 Jun 08 22:52:03.411418 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:03.411059 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sq5p4" event={"ID":"a03609a4-578a-4cca-80da-563aa1fb2aa7","Type":"ContainerDied","Data":"1baaaebaa3d406101fba17338b6c99418d237137b643704c090b2ad0b596bf88"} Jun 08 22:52:04.051087 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:04.051056 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:04.137196 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:04.137166 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:04.245508 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:04.245473 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:04.416520 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:04.416486 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sq5p4" event={"ID":"a03609a4-578a-4cca-80da-563aa1fb2aa7","Type":"ContainerStarted","Data":"3290c86559be13dc7e71906c2b0220a75b147d407d087bbbddb4f34d17f0c102"} Jun 08 22:52:04.416887 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:04.416542 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sq5p4" event={"ID":"a03609a4-578a-4cca-80da-563aa1fb2aa7","Type":"ContainerStarted","Data":"4cf83b6619841b3cb9bf7969a2244d3ff7b7e83c684dce64ec7296a261dd31c1"} Jun 08 22:52:04.431896 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:04.431855 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sq5p4" podStartSLOduration=2.420645172 podStartE2EDuration="3.431840207s" totalImagesPullingTime="945.563001ms" totalInitContainerRuntime="65.632034ms" isStatefulPod=true podCreationTimestamp="2026-06-08 22:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:52:04.431492082 +0000 UTC m=+246.240103935" watchObservedRunningTime="2026-06-08 22:52:04.431840207 +0000 UTC m=+246.240452059" Jun 08 22:52:04.553748 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:04.553712 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:05.811804 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:05.811774 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:05.838972 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:05.838943 2567 patch_prober.go:28] interesting pod/image-registry-ccd755cdc-76dgr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:52:05.839123 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:05.839007 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:52:05.839123 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:05.839047 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:52:05.839568 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:05.839548 2567 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"7d0a1edabafd0116548d2db5150926aaf697a20725f3dbf98ba78c113f96ffc5"} pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" containerMessage="Container registry failed liveness probe, will be restarted" Jun 08 22:52:05.843220 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:05.843196 2567 patch_prober.go:28] interesting pod/image-registry-ccd755cdc-76dgr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:52:05.843346 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:05.843238 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:52:05.953069 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:05.953037 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:06.052920 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:06.052889 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:06.110429 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:06.110326 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:07.282937 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:07.282896 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:08.160934 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:08.160860 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:10.481963 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:10.481928 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:52:10.484175 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:10.484146 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb0f393-3467-4695-8c0f-d31e1f735f9c-metrics-certs\") pod \"network-metrics-daemon-rds9j\" (UID: \"fbb0f393-3467-4695-8c0f-d31e1f735f9c\") " pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:52:10.520068 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:10.520046 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wjrnx\"" Jun 08 22:52:10.528594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:10.528575 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rds9j" Jun 08 22:52:10.652334 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:10.652302 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rds9j"] Jun 08 22:52:10.656383 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:52:10.656358 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb0f393_3467_4695_8c0f_d31e1f735f9c.slice/crio-06124a129c13d661ecb56c2c86baac03bad839f12b722fecd2d28bf2ffd4dce9 WatchSource:0}: Error finding container 06124a129c13d661ecb56c2c86baac03bad839f12b722fecd2d28bf2ffd4dce9: Status 404 returned error can't find the container with id 06124a129c13d661ecb56c2c86baac03bad839f12b722fecd2d28bf2ffd4dce9 Jun 08 22:52:11.438933 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:11.438903 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rds9j" event={"ID":"fbb0f393-3467-4695-8c0f-d31e1f735f9c","Type":"ContainerStarted","Data":"06124a129c13d661ecb56c2c86baac03bad839f12b722fecd2d28bf2ffd4dce9"} Jun 08 22:52:12.443603 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:12.443568 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rds9j" event={"ID":"fbb0f393-3467-4695-8c0f-d31e1f735f9c","Type":"ContainerStarted","Data":"65b65a3e620e8bddf5a69f3f34a1991e43ae5bca1305bc350ade1e6a52ebbc40"} Jun 08 22:52:12.443976 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:12.443628 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rds9j" event={"ID":"fbb0f393-3467-4695-8c0f-d31e1f735f9c","Type":"ContainerStarted","Data":"40c9d4027f01fa8bab3a4bb87554c9ea7867c5f056516c49641daca124f62dfd"} Jun 08 22:52:12.468492 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:12.468441 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rds9j" podStartSLOduration=253.638826179 podStartE2EDuration="4m14.468426388s" totalImagesPullingTime="829.600209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:52:12.467902879 +0000 UTC m=+254.276514735" watchObservedRunningTime="2026-06-08 22:52:12.468426388 +0000 UTC m=+254.277038242" Jun 08 22:52:15.843872 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:15.843839 2567 patch_prober.go:28] interesting pod/image-registry-ccd755cdc-76dgr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:52:15.844264 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:15.843890 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:52:16.272619 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:16.272568 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:16.317890 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:16.317858 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:17.385886 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:17.385850 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:25.843199 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:25.843160 2567 patch_prober.go:28] interesting pod/image-registry-ccd755cdc-76dgr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:52:25.843576 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:25.843214 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:52:25.847095 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:25.847064 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:25.948463 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:25.948434 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:28.972038 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:28.972007 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-f946b9758-8rw6s"] Jun 08 22:52:28.975135 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:28.975118 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:28.977642 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:28.977615 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 08 22:52:28.977757 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:28.977646 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 22:52:28.978080 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:28.978062 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3vjl3npsd97f6\"" Jun 08 22:52:28.978735 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:28.978508 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 08 22:52:28.978735 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:28.978577 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 08 22:52:28.979214 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:28.979072 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bkpkb\"" Jun 08 22:52:28.987821 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:28.987796 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-f946b9758-8rw6s"] Jun 08 22:52:29.013986 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.013951 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c00c525e-e22c-4d2e-abaa-da2c72c78a69-secret-metrics-server-tls\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.013986 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.013988 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c00c525e-e22c-4d2e-abaa-da2c72c78a69-audit-log\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.014208 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.014011 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c00c525e-e22c-4d2e-abaa-da2c72c78a69-client-ca-bundle\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.014208 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.014087 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c00c525e-e22c-4d2e-abaa-da2c72c78a69-secret-metrics-server-client-certs\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.014208 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.014163 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c00c525e-e22c-4d2e-abaa-da2c72c78a69-metrics-server-audit-profiles\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.014208 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.014192 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c00c525e-e22c-4d2e-abaa-da2c72c78a69-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.014349 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.014209 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjfbs\" (UniqueName: \"kubernetes.io/projected/c00c525e-e22c-4d2e-abaa-da2c72c78a69-kube-api-access-mjfbs\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.065744 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.065710 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:29.115371 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.115335 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c00c525e-e22c-4d2e-abaa-da2c72c78a69-secret-metrics-server-client-certs\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.115532 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.115382 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c00c525e-e22c-4d2e-abaa-da2c72c78a69-metrics-server-audit-profiles\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.115532 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.115425 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c00c525e-e22c-4d2e-abaa-da2c72c78a69-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.115532 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.115442 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjfbs\" (UniqueName: \"kubernetes.io/projected/c00c525e-e22c-4d2e-abaa-da2c72c78a69-kube-api-access-mjfbs\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.115532 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.115486 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c00c525e-e22c-4d2e-abaa-da2c72c78a69-secret-metrics-server-tls\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.115532 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.115502 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c00c525e-e22c-4d2e-abaa-da2c72c78a69-audit-log\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.115532 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.115522 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c00c525e-e22c-4d2e-abaa-da2c72c78a69-client-ca-bundle\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.116063 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.116032 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c00c525e-e22c-4d2e-abaa-da2c72c78a69-audit-log\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.116360 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.116335 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c00c525e-e22c-4d2e-abaa-da2c72c78a69-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.116715 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.116689 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c00c525e-e22c-4d2e-abaa-da2c72c78a69-metrics-server-audit-profiles\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.117983 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.117955 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c00c525e-e22c-4d2e-abaa-da2c72c78a69-client-ca-bundle\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.118066 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.118003 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c00c525e-e22c-4d2e-abaa-da2c72c78a69-secret-metrics-server-client-certs\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.118066 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.118059 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c00c525e-e22c-4d2e-abaa-da2c72c78a69-secret-metrics-server-tls\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.123171 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.123151 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjfbs\" (UniqueName: \"kubernetes.io/projected/c00c525e-e22c-4d2e-abaa-da2c72c78a69-kube-api-access-mjfbs\") pod \"metrics-server-f946b9758-8rw6s\" (UID: \"c00c525e-e22c-4d2e-abaa-da2c72c78a69\") " pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.155522 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.155494 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:29.247528 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.247445 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:29.288655 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.288620 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:29.413915 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.413883 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-f946b9758-8rw6s"] Jun 08 22:52:29.417163 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:52:29.417136 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00c525e_e22c_4d2e_abaa_da2c72c78a69.slice/crio-8888ee0510f6080523b73a3320a7f73c57360c881f362fe325bd2b04f8a78217 WatchSource:0}: Error finding container 8888ee0510f6080523b73a3320a7f73c57360c881f362fe325bd2b04f8a78217: Status 404 returned error can't find the container with id 8888ee0510f6080523b73a3320a7f73c57360c881f362fe325bd2b04f8a78217 Jun 08 22:52:29.497210 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:29.497177 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" event={"ID":"c00c525e-e22c-4d2e-abaa-da2c72c78a69","Type":"ContainerStarted","Data":"8888ee0510f6080523b73a3320a7f73c57360c881f362fe325bd2b04f8a78217"} Jun 08 22:52:30.857968 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:30.857938 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" containerID="cri-o://7d0a1edabafd0116548d2db5150926aaf697a20725f3dbf98ba78c113f96ffc5" gracePeriod=30 Jun 08 22:52:31.506298 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:31.506261 2567 generic.go:356] "Generic (PLEG): container finished" podID="06432f82-5309-4416-add3-3b0b77e7397e" containerID="7d0a1edabafd0116548d2db5150926aaf697a20725f3dbf98ba78c113f96ffc5" exitCode=0 Jun 08 22:52:31.506499 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:31.506327 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" event={"ID":"06432f82-5309-4416-add3-3b0b77e7397e","Type":"ContainerDied","Data":"7d0a1edabafd0116548d2db5150926aaf697a20725f3dbf98ba78c113f96ffc5"} Jun 08 22:52:31.506499 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:31.506384 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" event={"ID":"06432f82-5309-4416-add3-3b0b77e7397e","Type":"ContainerStarted","Data":"50bcc2ed5467922975b27ce9c9d3fe7fdd0091cf79b61d88e9b92c5a826b2c68"} Jun 08 22:52:31.506622 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:31.506568 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:52:31.507842 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:31.507821 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" event={"ID":"c00c525e-e22c-4d2e-abaa-da2c72c78a69","Type":"ContainerStarted","Data":"bbf757499aa23186180d93a089bc9d7d1354350d0882cfd269b845769d3107b7"} Jun 08 22:52:31.535612 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:31.535569 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" podStartSLOduration=2.151673077 podStartE2EDuration="3.535555665s" totalImagesPullingTime="1.383882588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:52:31.534460421 +0000 UTC m=+273.343072275" watchObservedRunningTime="2026-06-08 22:52:31.535555665 +0000 UTC m=+273.344167534" Jun 08 22:52:35.209200 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:35.209166 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:39.144679 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:52:39.144629 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9dmt5" podUID="1934d5fb-ba1e-48ec-86ee-ae2e12707285" Jun 08 22:52:39.539672 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:39.539638 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9dmt5" Jun 08 22:52:41.364201 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:41.364173 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-f946b9758-8rw6s_c00c525e-e22c-4d2e-abaa-da2c72c78a69/metrics-server/0.log" Jun 08 22:52:42.964472 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:42.964447 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq5p4_a03609a4-578a-4cca-80da-563aa1fb2aa7/init-textfile/0.log" Jun 08 22:52:43.029666 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:43.029629 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:52:43.031847 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:43.031817 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1934d5fb-ba1e-48ec-86ee-ae2e12707285-metrics-tls\") pod \"dns-default-9dmt5\" (UID: \"1934d5fb-ba1e-48ec-86ee-ae2e12707285\") " pod="openshift-dns/dns-default-9dmt5" Jun 08 22:52:43.142419 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:43.142368 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jnlbh\"" Jun 08 22:52:43.150478 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:43.150444 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9dmt5" Jun 08 22:52:43.165137 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:43.165114 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq5p4_a03609a4-578a-4cca-80da-563aa1fb2aa7/node-exporter/0.log" Jun 08 22:52:43.272638 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:43.272607 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9dmt5"] Jun 08 22:52:43.275785 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:52:43.275753 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1934d5fb_ba1e_48ec_86ee_ae2e12707285.slice/crio-840552177ae6ccbf545305a5e484323226a127185e380551edde380e4adce233 WatchSource:0}: Error finding container 840552177ae6ccbf545305a5e484323226a127185e380551edde380e4adce233: Status 404 returned error can't find the container with id 840552177ae6ccbf545305a5e484323226a127185e380551edde380e4adce233 Jun 08 22:52:43.364969 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:43.364935 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq5p4_a03609a4-578a-4cca-80da-563aa1fb2aa7/kube-rbac-proxy/0.log" Jun 08 22:52:43.553361 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:43.553272 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9dmt5" event={"ID":"1934d5fb-ba1e-48ec-86ee-ae2e12707285","Type":"ContainerStarted","Data":"840552177ae6ccbf545305a5e484323226a127185e380551edde380e4adce233"} Jun 08 22:52:44.755581 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:44.754711 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:44.856012 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:44.855980 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:44.955488 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:44.955463 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:45.121367 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:45.121289 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:45.221501 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:45.221456 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:45.562375 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:45.562334 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9dmt5" event={"ID":"1934d5fb-ba1e-48ec-86ee-ae2e12707285","Type":"ContainerStarted","Data":"be8c2cfe9672b08cbd94321600bb486bddceff83f6d7658b3c1781355b5fc0c9"} Jun 08 22:52:45.562566 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:45.562402 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9dmt5" event={"ID":"1934d5fb-ba1e-48ec-86ee-ae2e12707285","Type":"ContainerStarted","Data":"632c601ac8628ae8fd3d342bce4c357094a98b1fbf904bba726b59965b11687d"} Jun 08 22:52:45.562566 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:45.562445 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9dmt5" Jun 08 22:52:45.577061 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:45.577013 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9dmt5" podStartSLOduration=251.044465177 podStartE2EDuration="4m12.57699981s" totalImagesPullingTime="1.532534633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:52:45.576758813 +0000 UTC m=+287.385370667" watchObservedRunningTime="2026-06-08 22:52:45.57699981 +0000 UTC m=+287.385611663" Jun 08 22:52:49.164700 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:49.164670 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9dmt5_1934d5fb-ba1e-48ec-86ee-ae2e12707285/dns/0.log" Jun 08 22:52:49.288828 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:49.288791 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:49.288983 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:49.288848 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:52:49.364942 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:49.364914 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9dmt5_1934d5fb-ba1e-48ec-86ee-ae2e12707285/kube-rbac-proxy/0.log" Jun 08 22:52:49.526752 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:49.526722 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:49.575714 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:49.575688 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:49.634704 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:49.634679 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:49.732885 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:49.732855 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:49.964719 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:49.964687 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gpjcx_f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e/dns-node-resolver/0.log" Jun 08 22:52:50.964944 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:50.964917 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-ccd755cdc-76dgr_06432f82-5309-4416-add3-3b0b77e7397e/registry/0.log" Jun 08 22:52:51.165277 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:51.165250 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-ccd755cdc-76dgr_06432f82-5309-4416-add3-3b0b77e7397e/registry/1.log" Jun 08 22:52:51.364077 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:51.364013 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c45cn_b8c10f1b-1d19-4d08-b676-8434b760c799/node-ca/0.log" Jun 08 22:52:52.521643 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:52.521616 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:52:52.565236 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:52.565208 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z79nn_3fa35cc0-fc3e-467e-b9a8-16da413b82ed/serve-healthcheck-canary/0.log" Jun 08 22:52:52.678126 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:52.678071 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:52.779315 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:52.779232 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:52.879902 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:52.879868 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:52.926839 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:52.926808 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:55.521412 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:55.521362 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ccd755cdc-76dgr"] Jun 08 22:52:55.568117 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:55.568084 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9dmt5" Jun 08 22:52:55.603929 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:55.603883 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:55.731900 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:55.731868 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:55.833947 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:55.833875 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:56.935088 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:56.935057 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:57.137005 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:57.136972 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:52:58.565527 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:58.565496 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 22:52:58.566458 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:58.566433 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 22:52:58.570568 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:58.570538 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 22:52:58.571321 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:58.571290 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 22:52:58.572676 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:52:58.572659 2567 kubelet.go:1648] "Image garbage collection succeeded" Jun 08 22:53:01.906526 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:01.906481 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:01.951334 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:01.951304 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:04.684971 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:04.684939 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:04.786064 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:04.786027 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:09.294764 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:09.294736 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:53:09.298565 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:09.298548 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-f946b9758-8rw6s" Jun 08 22:53:11.938578 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:11.938541 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:11.987830 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:11.987796 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:13.411519 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:13.411483 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:13.512862 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:13.512830 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:13.561121 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:13.561093 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:13.683898 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:13.683864 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:13.784320 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:13.784288 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:20.541044 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:20.541004 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" containerID="cri-o://50bcc2ed5467922975b27ce9c9d3fe7fdd0091cf79b61d88e9b92c5a826b2c68" gracePeriod=30 Jun 08 22:53:21.690211 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.690181 2567 generic.go:356] "Generic (PLEG): container finished" podID="06432f82-5309-4416-add3-3b0b77e7397e" containerID="50bcc2ed5467922975b27ce9c9d3fe7fdd0091cf79b61d88e9b92c5a826b2c68" exitCode=0 Jun 08 22:53:21.690653 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.690229 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" event={"ID":"06432f82-5309-4416-add3-3b0b77e7397e","Type":"ContainerDied","Data":"50bcc2ed5467922975b27ce9c9d3fe7fdd0091cf79b61d88e9b92c5a826b2c68"} Jun 08 22:53:21.690653 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.690285 2567 scope.go:117] "RemoveContainer" containerID="7d0a1edabafd0116548d2db5150926aaf697a20725f3dbf98ba78c113f96ffc5" Jun 08 22:53:21.782830 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.782801 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:53:21.806872 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.806846 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-trusted-ca\") pod \"06432f82-5309-4416-add3-3b0b77e7397e\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " Jun 08 22:53:21.806872 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.806878 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-bound-sa-token\") pod \"06432f82-5309-4416-add3-3b0b77e7397e\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " Jun 08 22:53:21.807097 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.806905 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-registry-certificates\") pod \"06432f82-5309-4416-add3-3b0b77e7397e\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " Jun 08 22:53:21.807097 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.806923 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-installation-pull-secrets\") pod \"06432f82-5309-4416-add3-3b0b77e7397e\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " Jun 08 22:53:21.807097 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.806953 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ss55\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-kube-api-access-7ss55\") pod \"06432f82-5309-4416-add3-3b0b77e7397e\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " Jun 08 22:53:21.807097 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.806984 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-registry-tls\") pod \"06432f82-5309-4416-add3-3b0b77e7397e\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " Jun 08 22:53:21.807097 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.807011 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06432f82-5309-4416-add3-3b0b77e7397e-ca-trust-extracted\") pod \"06432f82-5309-4416-add3-3b0b77e7397e\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " Jun 08 22:53:21.807097 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.807042 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-image-registry-private-configuration\") pod \"06432f82-5309-4416-add3-3b0b77e7397e\" (UID: \"06432f82-5309-4416-add3-3b0b77e7397e\") " Jun 08 22:53:21.807441 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.807412 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "06432f82-5309-4416-add3-3b0b77e7397e" (UID: "06432f82-5309-4416-add3-3b0b77e7397e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:53:21.807441 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.807426 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "06432f82-5309-4416-add3-3b0b77e7397e" (UID: "06432f82-5309-4416-add3-3b0b77e7397e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:53:21.809961 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.809927 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "06432f82-5309-4416-add3-3b0b77e7397e" (UID: "06432f82-5309-4416-add3-3b0b77e7397e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:53:21.810093 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.809957 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "06432f82-5309-4416-add3-3b0b77e7397e" (UID: "06432f82-5309-4416-add3-3b0b77e7397e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:53:21.810093 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.809963 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "06432f82-5309-4416-add3-3b0b77e7397e" (UID: "06432f82-5309-4416-add3-3b0b77e7397e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:53:21.810093 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.809935 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "06432f82-5309-4416-add3-3b0b77e7397e" (UID: "06432f82-5309-4416-add3-3b0b77e7397e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:53:21.810093 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.810076 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-kube-api-access-7ss55" (OuterVolumeSpecName: "kube-api-access-7ss55") pod "06432f82-5309-4416-add3-3b0b77e7397e" (UID: "06432f82-5309-4416-add3-3b0b77e7397e"). InnerVolumeSpecName "kube-api-access-7ss55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:53:21.818525 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.818496 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06432f82-5309-4416-add3-3b0b77e7397e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "06432f82-5309-4416-add3-3b0b77e7397e" (UID: "06432f82-5309-4416-add3-3b0b77e7397e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:53:21.908183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.908099 2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-bound-sa-token\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:53:21.908183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.908127 2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-registry-certificates\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:53:21.908183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.908138 2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-installation-pull-secrets\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:53:21.908183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.908147 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ss55\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-kube-api-access-7ss55\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:53:21.908183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.908156 2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/06432f82-5309-4416-add3-3b0b77e7397e-registry-tls\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:53:21.908183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.908164 2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/06432f82-5309-4416-add3-3b0b77e7397e-ca-trust-extracted\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:53:21.908183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.908173 2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/06432f82-5309-4416-add3-3b0b77e7397e-image-registry-private-configuration\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:53:21.908183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:21.908182 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/06432f82-5309-4416-add3-3b0b77e7397e-trusted-ca\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:53:22.694021 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:22.693980 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" event={"ID":"06432f82-5309-4416-add3-3b0b77e7397e","Type":"ContainerDied","Data":"10e3b6a59f2b73e9a994ceb9f227cb10f75d60c4d174d8c31cb796df82c251a9"} Jun 08 22:53:22.694021 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:22.694007 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ccd755cdc-76dgr" Jun 08 22:53:22.694523 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:22.694051 2567 scope.go:117] "RemoveContainer" containerID="50bcc2ed5467922975b27ce9c9d3fe7fdd0091cf79b61d88e9b92c5a826b2c68" Jun 08 22:53:22.708438 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:22.708413 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ccd755cdc-76dgr"] Jun 08 22:53:22.712090 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:22.712068 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-ccd755cdc-76dgr"] Jun 08 22:53:24.619789 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:24.619756 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06432f82-5309-4416-add3-3b0b77e7397e" path="/var/lib/kubelet/pods/06432f82-5309-4416-add3-3b0b77e7397e/volumes" Jun 08 22:53:26.934725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:26.934696 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:27.116468 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:27.116437 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:40.283276 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:40.283204 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:40.700083 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:40.700051 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:45.364858 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.364816 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-54845cbdfc-wg854"] Jun 08 22:53:45.365245 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.365157 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" Jun 08 22:53:45.365245 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.365176 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" Jun 08 22:53:45.365245 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.365197 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" Jun 08 22:53:45.365245 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.365207 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" Jun 08 22:53:45.365374 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.365268 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" Jun 08 22:53:45.365374 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.365338 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="06432f82-5309-4416-add3-3b0b77e7397e" containerName="registry" Jun 08 22:53:45.368112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.368097 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.370064 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.370038 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 08 22:53:45.370190 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.370047 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 08 22:53:45.370190 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.370045 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 08 22:53:45.370190 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.370045 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 08 22:53:45.370325 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.370185 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8m5vx\"" Jun 08 22:53:45.370325 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.370322 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 08 22:53:45.375509 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.375492 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 08 22:53:45.379135 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.379115 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54845cbdfc-wg854"] Jun 08 22:53:45.449727 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.449693 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-telemeter-client-tls\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.449895 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.449739 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.449895 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.449766 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-secret-telemeter-client\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.449895 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.449787 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xns9p\" (UniqueName: \"kubernetes.io/projected/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-kube-api-access-xns9p\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.449895 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.449816 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-serving-certs-ca-bundle\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.449895 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.449869 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-metrics-client-ca\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.450079 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.449916 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.450079 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.449939 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-federate-client-tls\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.551194 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.551144 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.551194 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.551198 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-federate-client-tls\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.551472 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.551227 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-telemeter-client-tls\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.551472 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.551256 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.551472 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.551379 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-secret-telemeter-client\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.551472 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.551445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xns9p\" (UniqueName: \"kubernetes.io/projected/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-kube-api-access-xns9p\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.551677 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.551491 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-serving-certs-ca-bundle\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.551677 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.551525 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-metrics-client-ca\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.552175 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.552151 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-serving-certs-ca-bundle\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.552277 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.552199 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-metrics-client-ca\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.552277 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.552236 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.554320 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.554291 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-federate-client-tls\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.554427 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.554296 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.554477 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.554432 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-secret-telemeter-client\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.554477 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.554446 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-telemeter-client-tls\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.558363 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.558345 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xns9p\" (UniqueName: \"kubernetes.io/projected/3ca7e03a-3d4d-4084-acd7-002b7adc5d04-kube-api-access-xns9p\") pod \"telemeter-client-54845cbdfc-wg854\" (UID: \"3ca7e03a-3d4d-4084-acd7-002b7adc5d04\") " pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.678070 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.678039 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" Jun 08 22:53:45.803201 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.803171 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54845cbdfc-wg854"] Jun 08 22:53:45.806006 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:53:45.805976 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca7e03a_3d4d_4084_acd7_002b7adc5d04.slice/crio-7765f79a7c4f56308148ca02c11e5cffdef04a2d223b0f99ab575d7ec3f803b2 WatchSource:0}: Error finding container 7765f79a7c4f56308148ca02c11e5cffdef04a2d223b0f99ab575d7ec3f803b2: Status 404 returned error can't find the container with id 7765f79a7c4f56308148ca02c11e5cffdef04a2d223b0f99ab575d7ec3f803b2 Jun 08 22:53:45.807840 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:45.807823 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 22:53:46.774899 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:46.774838 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" event={"ID":"3ca7e03a-3d4d-4084-acd7-002b7adc5d04","Type":"ContainerStarted","Data":"7765f79a7c4f56308148ca02c11e5cffdef04a2d223b0f99ab575d7ec3f803b2"} Jun 08 22:53:48.783319 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:48.783279 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" event={"ID":"3ca7e03a-3d4d-4084-acd7-002b7adc5d04","Type":"ContainerStarted","Data":"cdb83fb90b91f579a13b50454db4ab798c44cef98d2aae0945ed0a3455de7672"} Jun 08 22:53:49.788641 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:49.788608 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" event={"ID":"3ca7e03a-3d4d-4084-acd7-002b7adc5d04","Type":"ContainerStarted","Data":"40a3ed1c960b607ffb77da18653eabaaef981c98f279c95bc474ad0255ad94c3"} Jun 08 22:53:49.789024 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:49.788659 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" event={"ID":"3ca7e03a-3d4d-4084-acd7-002b7adc5d04","Type":"ContainerStarted","Data":"bd42d8cca6109a735f4ad7063acadf2beb7a1efcbd7f0245de174f04fa874a9c"} Jun 08 22:53:49.809312 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:49.809250 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-54845cbdfc-wg854" podStartSLOduration=1.5817653489999999 podStartE2EDuration="4.809231558s" totalImagesPullingTime="3.227466209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:53:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:53:49.808340261 +0000 UTC m=+351.616952114" watchObservedRunningTime="2026-06-08 22:53:49.809231558 +0000 UTC m=+351.617843415" Jun 08 22:53:50.768348 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:50.768310 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:50.943010 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:50.942968 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:50.989930 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:50.989902 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:51.520497 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:51.520452 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:53:51.568106 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:53:51.568076 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:01.950938 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:01.950897 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:02.008386 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:02.008355 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:02.065012 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:02.064987 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:07.082295 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:07.082261 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:08.403333 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:08.403300 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:12.627659 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:12.627628 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:14.420544 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:14.420515 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:20.130115 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:20.130086 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:55.456778 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:55.456742 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:55.504275 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:55.504225 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:56.860064 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:56.860036 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:56.904155 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:56.904124 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:57.015683 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:57.015649 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:57.116928 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:57.116845 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:57.605563 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:57.605532 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:54:57.649663 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:54:57.649636 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:07.334032 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:07.333999 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:07.381054 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:07.381030 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:08.198956 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:08.198889 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:08.247278 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:08.247247 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:08.304867 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:08.304844 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:11.939615 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:11.939584 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:12.040561 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:12.040532 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:12.221138 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:12.221061 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:50.814678 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:50.814642 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:50.912788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:50.912757 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:55:51.292450 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.292411 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m"] Jun 08 22:55:51.295593 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.295572 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.297885 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.297863 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7qzv5\"" Jun 08 22:55:51.297976 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.297909 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 08 22:55:51.297976 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.297908 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 08 22:55:51.303767 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.303748 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m"] Jun 08 22:55:51.339743 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.339709 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.339883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.339749 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.339883 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.339771 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr5wb\" (UniqueName: \"kubernetes.io/projected/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-kube-api-access-hr5wb\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.441076 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.441045 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.441219 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.441084 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.441219 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.441114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr5wb\" (UniqueName: \"kubernetes.io/projected/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-kube-api-access-hr5wb\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.441548 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.441528 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.441591 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.441530 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.449215 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.449189 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr5wb\" (UniqueName: \"kubernetes.io/projected/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-kube-api-access-hr5wb\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.604691 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.604588 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:55:51.725654 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:51.725628 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m"] Jun 08 22:55:51.728214 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:55:51.728188 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc19737_bcba_401d_ae48_c26fe9fe0e7c.slice/crio-2eb4737c1e660895898682ca63922f0434263cecbf2d6967bf468fe872001f6b WatchSource:0}: Error finding container 2eb4737c1e660895898682ca63922f0434263cecbf2d6967bf468fe872001f6b: Status 404 returned error can't find the container with id 2eb4737c1e660895898682ca63922f0434263cecbf2d6967bf468fe872001f6b Jun 08 22:55:52.193410 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:52.193358 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" event={"ID":"6bc19737-bcba-401d-ae48-c26fe9fe0e7c","Type":"ContainerStarted","Data":"2eb4737c1e660895898682ca63922f0434263cecbf2d6967bf468fe872001f6b"} Jun 08 22:55:58.218458 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:58.218418 2567 generic.go:356] "Generic (PLEG): container finished" podID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerID="834b22f064e03bf5d67054b22f661271acc4a07d328fd096140615a5805b251e" exitCode=0 Jun 08 22:55:58.218815 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:55:58.218518 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" event={"ID":"6bc19737-bcba-401d-ae48-c26fe9fe0e7c","Type":"ContainerDied","Data":"834b22f064e03bf5d67054b22f661271acc4a07d328fd096140615a5805b251e"} Jun 08 22:56:00.226091 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:00.226058 2567 generic.go:356] "Generic (PLEG): container finished" podID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerID="1488c344cb985b325370ed5298042eeddd63d5615664311ca736cad481a072c1" exitCode=0 Jun 08 22:56:00.226485 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:00.226140 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" event={"ID":"6bc19737-bcba-401d-ae48-c26fe9fe0e7c","Type":"ContainerDied","Data":"1488c344cb985b325370ed5298042eeddd63d5615664311ca736cad481a072c1"} Jun 08 22:56:07.256058 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:07.256021 2567 generic.go:356] "Generic (PLEG): container finished" podID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerID="b07926edee05946745da73387022fd3fc7c30f0cbd7e0598b35e551dc64bc36c" exitCode=0 Jun 08 22:56:07.256505 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:07.256084 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" event={"ID":"6bc19737-bcba-401d-ae48-c26fe9fe0e7c","Type":"ContainerDied","Data":"b07926edee05946745da73387022fd3fc7c30f0cbd7e0598b35e551dc64bc36c"} Jun 08 22:56:08.378148 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.378123 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:56:08.479324 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.479275 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-util\") pod \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " Jun 08 22:56:08.479522 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.479354 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr5wb\" (UniqueName: \"kubernetes.io/projected/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-kube-api-access-hr5wb\") pod \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " Jun 08 22:56:08.479522 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.479380 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-bundle\") pod \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\" (UID: \"6bc19737-bcba-401d-ae48-c26fe9fe0e7c\") " Jun 08 22:56:08.480062 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.480035 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-bundle" (OuterVolumeSpecName: "bundle") pod "6bc19737-bcba-401d-ae48-c26fe9fe0e7c" (UID: "6bc19737-bcba-401d-ae48-c26fe9fe0e7c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:56:08.481564 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.481534 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-kube-api-access-hr5wb" (OuterVolumeSpecName: "kube-api-access-hr5wb") pod "6bc19737-bcba-401d-ae48-c26fe9fe0e7c" (UID: "6bc19737-bcba-401d-ae48-c26fe9fe0e7c"). InnerVolumeSpecName "kube-api-access-hr5wb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:56:08.483243 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.483219 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-util" (OuterVolumeSpecName: "util") pod "6bc19737-bcba-401d-ae48-c26fe9fe0e7c" (UID: "6bc19737-bcba-401d-ae48-c26fe9fe0e7c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:56:08.580966 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.580881 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:56:08.580966 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.580914 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr5wb\" (UniqueName: \"kubernetes.io/projected/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-kube-api-access-hr5wb\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:56:08.580966 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:08.580925 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bc19737-bcba-401d-ae48-c26fe9fe0e7c-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:56:09.264802 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:09.264766 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" event={"ID":"6bc19737-bcba-401d-ae48-c26fe9fe0e7c","Type":"ContainerDied","Data":"2eb4737c1e660895898682ca63922f0434263cecbf2d6967bf468fe872001f6b"} Jun 08 22:56:09.264964 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:09.264813 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dspf9m" Jun 08 22:56:09.264964 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:09.264833 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2eb4737c1e660895898682ca63922f0434263cecbf2d6967bf468fe872001f6b" Jun 08 22:56:12.221198 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:12.221167 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:13.028220 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:13.028190 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:15.620927 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:15.620895 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:16.628749 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:16.628710 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:18.415480 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.415450 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:18.530990 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.530957 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:18.717781 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.717749 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:18.812744 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.812701 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:18.884164 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.884131 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t"] Jun 08 22:56:18.884411 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.884376 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerName="util" Jun 08 22:56:18.884411 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.884403 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerName="util" Jun 08 22:56:18.884502 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.884415 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerName="pull" Jun 08 22:56:18.884502 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.884422 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerName="pull" Jun 08 22:56:18.884502 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.884437 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerName="extract" Jun 08 22:56:18.884502 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.884443 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerName="extract" Jun 08 22:56:18.884502 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.884480 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc19737-bcba-401d-ae48-c26fe9fe0e7c" containerName="extract" Jun 08 22:56:18.886447 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.886431 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:18.889121 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.889101 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7qzv5\"" Jun 08 22:56:18.889225 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.889144 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 08 22:56:18.890901 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.890886 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 08 22:56:18.897177 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.897156 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t"] Jun 08 22:56:18.956284 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.956254 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:18.956477 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.956310 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:18.956477 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:18.956331 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhpgk\" (UniqueName: \"kubernetes.io/projected/824a884f-38d4-48c8-bcce-c0e8197fca02-kube-api-access-nhpgk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:19.056884 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:19.056785 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:19.056884 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:19.056864 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:19.057061 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:19.056896 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhpgk\" (UniqueName: \"kubernetes.io/projected/824a884f-38d4-48c8-bcce-c0e8197fca02-kube-api-access-nhpgk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:19.057302 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:19.057282 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:19.057334 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:19.057304 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:19.064262 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:19.064235 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhpgk\" (UniqueName: \"kubernetes.io/projected/824a884f-38d4-48c8-bcce-c0e8197fca02-kube-api-access-nhpgk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:19.195764 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:19.195732 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:19.319219 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:19.319164 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t"] Jun 08 22:56:19.321711 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:56:19.321681 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod824a884f_38d4_48c8_bcce_c0e8197fca02.slice/crio-728755c0e5a3f5d6bd8ec8f16425f21a498b6fc51923bf679d05e41e6fabdb2f WatchSource:0}: Error finding container 728755c0e5a3f5d6bd8ec8f16425f21a498b6fc51923bf679d05e41e6fabdb2f: Status 404 returned error can't find the container with id 728755c0e5a3f5d6bd8ec8f16425f21a498b6fc51923bf679d05e41e6fabdb2f Jun 08 22:56:20.303873 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:20.303836 2567 generic.go:356] "Generic (PLEG): container finished" podID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerID="5e444c45a8f9c65d386bc79e1eefade4ac271983c079dd6d7222669582acc512" exitCode=0 Jun 08 22:56:20.304246 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:20.303900 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" event={"ID":"824a884f-38d4-48c8-bcce-c0e8197fca02","Type":"ContainerDied","Data":"5e444c45a8f9c65d386bc79e1eefade4ac271983c079dd6d7222669582acc512"} Jun 08 22:56:20.304246 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:20.303945 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" event={"ID":"824a884f-38d4-48c8-bcce-c0e8197fca02","Type":"ContainerStarted","Data":"728755c0e5a3f5d6bd8ec8f16425f21a498b6fc51923bf679d05e41e6fabdb2f"} Jun 08 22:56:21.714984 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:21.714944 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:22.306151 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:22.306121 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:22.424737 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:22.424696 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:22.524565 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:22.524518 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:23.316920 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:23.316885 2567 generic.go:356] "Generic (PLEG): container finished" podID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerID="946e0458b6f5fed9db53cca2cf416e3c4dd7124916598df4d1b986b6b4020b9f" exitCode=0 Jun 08 22:56:23.317288 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:23.316931 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" event={"ID":"824a884f-38d4-48c8-bcce-c0e8197fca02","Type":"ContainerDied","Data":"946e0458b6f5fed9db53cca2cf416e3c4dd7124916598df4d1b986b6b4020b9f"} Jun 08 22:56:24.322535 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:24.322503 2567 generic.go:356] "Generic (PLEG): container finished" podID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerID="15abf8a3402cbfd20ecb40518dd7ea7eb81feb8729252b50cde420dc01f8f6c4" exitCode=0 Jun 08 22:56:24.322535 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:24.322538 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" event={"ID":"824a884f-38d4-48c8-bcce-c0e8197fca02","Type":"ContainerDied","Data":"15abf8a3402cbfd20ecb40518dd7ea7eb81feb8729252b50cde420dc01f8f6c4"} Jun 08 22:56:25.450748 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.450720 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:25.509305 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.509263 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhpgk\" (UniqueName: \"kubernetes.io/projected/824a884f-38d4-48c8-bcce-c0e8197fca02-kube-api-access-nhpgk\") pod \"824a884f-38d4-48c8-bcce-c0e8197fca02\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " Jun 08 22:56:25.509488 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.509353 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-bundle\") pod \"824a884f-38d4-48c8-bcce-c0e8197fca02\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " Jun 08 22:56:25.509488 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.509374 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-util\") pod \"824a884f-38d4-48c8-bcce-c0e8197fca02\" (UID: \"824a884f-38d4-48c8-bcce-c0e8197fca02\") " Jun 08 22:56:25.509756 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.509731 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-bundle" (OuterVolumeSpecName: "bundle") pod "824a884f-38d4-48c8-bcce-c0e8197fca02" (UID: "824a884f-38d4-48c8-bcce-c0e8197fca02"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:56:25.511371 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.511346 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824a884f-38d4-48c8-bcce-c0e8197fca02-kube-api-access-nhpgk" (OuterVolumeSpecName: "kube-api-access-nhpgk") pod "824a884f-38d4-48c8-bcce-c0e8197fca02" (UID: "824a884f-38d4-48c8-bcce-c0e8197fca02"). InnerVolumeSpecName "kube-api-access-nhpgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:56:25.514029 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.514010 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-util" (OuterVolumeSpecName: "util") pod "824a884f-38d4-48c8-bcce-c0e8197fca02" (UID: "824a884f-38d4-48c8-bcce-c0e8197fca02"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:56:25.610722 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.610630 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhpgk\" (UniqueName: \"kubernetes.io/projected/824a884f-38d4-48c8-bcce-c0e8197fca02-kube-api-access-nhpgk\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:56:25.610722 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.610668 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:56:25.610722 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:25.610678 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824a884f-38d4-48c8-bcce-c0e8197fca02-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:56:26.095515 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.095476 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt"] Jun 08 22:56:26.095707 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.095696 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerName="util" Jun 08 22:56:26.095750 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.095709 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerName="util" Jun 08 22:56:26.095750 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.095724 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerName="pull" Jun 08 22:56:26.095750 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.095729 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerName="pull" Jun 08 22:56:26.095750 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.095740 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerName="extract" Jun 08 22:56:26.095750 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.095745 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerName="extract" Jun 08 22:56:26.095931 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.095790 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="824a884f-38d4-48c8-bcce-c0e8197fca02" containerName="extract" Jun 08 22:56:26.099805 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.099788 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" Jun 08 22:56:26.101696 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.101677 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 08 22:56:26.102164 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.102136 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 08 22:56:26.102246 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.102142 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9dpgm\"" Jun 08 22:56:26.105275 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.105250 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt"] Jun 08 22:56:26.214645 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.214608 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drrvg\" (UniqueName: \"kubernetes.io/projected/3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a-kube-api-access-drrvg\") pod \"cert-manager-cainjector-5d99449ffd-xvmgt\" (UID: \"3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" Jun 08 22:56:26.214828 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.214654 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-xvmgt\" (UID: \"3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" Jun 08 22:56:26.315190 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.315155 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drrvg\" (UniqueName: \"kubernetes.io/projected/3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a-kube-api-access-drrvg\") pod \"cert-manager-cainjector-5d99449ffd-xvmgt\" (UID: \"3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" Jun 08 22:56:26.315190 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.315192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-xvmgt\" (UID: \"3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" Jun 08 22:56:26.322637 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.322603 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-xvmgt\" (UID: \"3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" Jun 08 22:56:26.322768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.322750 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drrvg\" (UniqueName: \"kubernetes.io/projected/3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a-kube-api-access-drrvg\") pod \"cert-manager-cainjector-5d99449ffd-xvmgt\" (UID: \"3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" Jun 08 22:56:26.331402 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.331367 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" Jun 08 22:56:26.331509 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.331378 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmq58t" event={"ID":"824a884f-38d4-48c8-bcce-c0e8197fca02","Type":"ContainerDied","Data":"728755c0e5a3f5d6bd8ec8f16425f21a498b6fc51923bf679d05e41e6fabdb2f"} Jun 08 22:56:26.331509 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.331447 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="728755c0e5a3f5d6bd8ec8f16425f21a498b6fc51923bf679d05e41e6fabdb2f" Jun 08 22:56:26.409270 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.409179 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" Jun 08 22:56:26.528078 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:26.528052 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt"] Jun 08 22:56:26.530434 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:56:26.530410 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c9ff76d_c0c1_4b28_99bb_bba6b3274e1a.slice/crio-4f1ebf4db8e2e20277538ff961e683f10bcdecb143fa2efa2cca6cb009bcebc3 WatchSource:0}: Error finding container 4f1ebf4db8e2e20277538ff961e683f10bcdecb143fa2efa2cca6cb009bcebc3: Status 404 returned error can't find the container with id 4f1ebf4db8e2e20277538ff961e683f10bcdecb143fa2efa2cca6cb009bcebc3 Jun 08 22:56:27.336775 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:27.336737 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" event={"ID":"3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a","Type":"ContainerStarted","Data":"4f1ebf4db8e2e20277538ff961e683f10bcdecb143fa2efa2cca6cb009bcebc3"} Jun 08 22:56:29.347794 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:29.346994 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" event={"ID":"3c9ff76d-c0c1-4b28-99bb-bba6b3274e1a","Type":"ContainerStarted","Data":"aed6485fa44f3706d5c32c489794bcc4dc10992e65ae081b4c563ac7ed2372c1"} Jun 08 22:56:29.361533 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:29.361477 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-xvmgt" podStartSLOduration=0.932062401 podStartE2EDuration="3.36145498s" totalImagesPullingTime="2.429392579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:56:29.360176962 +0000 UTC m=+511.168788816" watchObservedRunningTime="2026-06-08 22:56:29.36145498 +0000 UTC m=+511.170066834" Jun 08 22:56:29.528264 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:29.528233 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:29.631071 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:29.630976 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:29.869962 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:29.869931 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:30.144968 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:30.144932 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:30.246322 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:30.246293 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:30.342446 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:30.342416 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:36.761448 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:36.761416 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:36.864008 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:36.863971 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:38.344929 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.344849 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk"] Jun 08 22:56:38.346937 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.346921 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.348714 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.348692 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 08 22:56:38.348858 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.348750 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7qzv5\"" Jun 08 22:56:38.349378 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.349356 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 08 22:56:38.355022 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.354990 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk"] Jun 08 22:56:38.450954 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.450915 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:38.514201 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.514154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.514383 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.514236 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q5r9\" (UniqueName: \"kubernetes.io/projected/8d122723-638d-46fd-ae09-d6cb6763ad4f-kube-api-access-4q5r9\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.514383 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.514270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.615232 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.615139 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q5r9\" (UniqueName: \"kubernetes.io/projected/8d122723-638d-46fd-ae09-d6cb6763ad4f-kube-api-access-4q5r9\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.615232 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.615199 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.615483 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.615246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.615729 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.615701 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.615768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.615721 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.625093 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.625066 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q5r9\" (UniqueName: \"kubernetes.io/projected/8d122723-638d-46fd-ae09-d6cb6763ad4f-kube-api-access-4q5r9\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.656559 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.656525 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:38.781475 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:38.781447 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk"] Jun 08 22:56:38.784011 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:56:38.783984 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d122723_638d_46fd_ae09_d6cb6763ad4f.slice/crio-b5aa853229f43eac13d0e6c9ee38779625e3842696bf4f32a31e27ced4406b6f WatchSource:0}: Error finding container b5aa853229f43eac13d0e6c9ee38779625e3842696bf4f32a31e27ced4406b6f: Status 404 returned error can't find the container with id b5aa853229f43eac13d0e6c9ee38779625e3842696bf4f32a31e27ced4406b6f Jun 08 22:56:39.382717 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:39.382685 2567 generic.go:356] "Generic (PLEG): container finished" podID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerID="0076a369c35c3f2ec3182c1a1e4e0cad8d6aa440c763881fa71f710c8eaefbd7" exitCode=0 Jun 08 22:56:39.383115 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:39.382758 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" event={"ID":"8d122723-638d-46fd-ae09-d6cb6763ad4f","Type":"ContainerDied","Data":"0076a369c35c3f2ec3182c1a1e4e0cad8d6aa440c763881fa71f710c8eaefbd7"} Jun 08 22:56:39.383115 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:39.382793 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" event={"ID":"8d122723-638d-46fd-ae09-d6cb6763ad4f","Type":"ContainerStarted","Data":"b5aa853229f43eac13d0e6c9ee38779625e3842696bf4f32a31e27ced4406b6f"} Jun 08 22:56:40.387836 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:40.387742 2567 generic.go:356] "Generic (PLEG): container finished" podID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerID="04cd6a7e8421f6c0e9727062f688a3dfd994c05bff0ddeaa4d8e1961a5eed117" exitCode=0 Jun 08 22:56:40.388248 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:40.387830 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" event={"ID":"8d122723-638d-46fd-ae09-d6cb6763ad4f","Type":"ContainerDied","Data":"04cd6a7e8421f6c0e9727062f688a3dfd994c05bff0ddeaa4d8e1961a5eed117"} Jun 08 22:56:41.393693 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:41.393663 2567 generic.go:356] "Generic (PLEG): container finished" podID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerID="e927882c9b55d49b79ed08a0de55967d7a2c21638ec2df31220fd94fe5c84d54" exitCode=0 Jun 08 22:56:41.393693 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:41.393696 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" event={"ID":"8d122723-638d-46fd-ae09-d6cb6763ad4f","Type":"ContainerDied","Data":"e927882c9b55d49b79ed08a0de55967d7a2c21638ec2df31220fd94fe5c84d54"} Jun 08 22:56:42.518282 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.518253 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:42.646044 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.646014 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-util\") pod \"8d122723-638d-46fd-ae09-d6cb6763ad4f\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " Jun 08 22:56:42.646044 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.646050 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q5r9\" (UniqueName: \"kubernetes.io/projected/8d122723-638d-46fd-ae09-d6cb6763ad4f-kube-api-access-4q5r9\") pod \"8d122723-638d-46fd-ae09-d6cb6763ad4f\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " Jun 08 22:56:42.646280 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.646143 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-bundle\") pod \"8d122723-638d-46fd-ae09-d6cb6763ad4f\" (UID: \"8d122723-638d-46fd-ae09-d6cb6763ad4f\") " Jun 08 22:56:42.646835 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.646802 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-bundle" (OuterVolumeSpecName: "bundle") pod "8d122723-638d-46fd-ae09-d6cb6763ad4f" (UID: "8d122723-638d-46fd-ae09-d6cb6763ad4f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:56:42.648193 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.648141 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d122723-638d-46fd-ae09-d6cb6763ad4f-kube-api-access-4q5r9" (OuterVolumeSpecName: "kube-api-access-4q5r9") pod "8d122723-638d-46fd-ae09-d6cb6763ad4f" (UID: "8d122723-638d-46fd-ae09-d6cb6763ad4f"). InnerVolumeSpecName "kube-api-access-4q5r9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:56:42.651469 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.651445 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-util" (OuterVolumeSpecName: "util") pod "8d122723-638d-46fd-ae09-d6cb6763ad4f" (UID: "8d122723-638d-46fd-ae09-d6cb6763ad4f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:56:42.747584 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.747544 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:56:42.747584 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.747578 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d122723-638d-46fd-ae09-d6cb6763ad4f-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:56:42.747584 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:42.747591 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q5r9\" (UniqueName: \"kubernetes.io/projected/8d122723-638d-46fd-ae09-d6cb6763ad4f-kube-api-access-4q5r9\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:56:43.402587 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:43.402557 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" Jun 08 22:56:43.402587 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:43.402571 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cd5kklk" event={"ID":"8d122723-638d-46fd-ae09-d6cb6763ad4f","Type":"ContainerDied","Data":"b5aa853229f43eac13d0e6c9ee38779625e3842696bf4f32a31e27ced4406b6f"} Jun 08 22:56:43.402758 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:43.402613 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5aa853229f43eac13d0e6c9ee38779625e3842696bf4f32a31e27ced4406b6f" Jun 08 22:56:54.509658 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:54.509602 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:54.605112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:54.605079 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:55.111530 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.111501 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:55.286756 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.286725 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:55.394095 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.394009 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:55.433267 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.433240 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz"] Jun 08 22:56:55.433517 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.433504 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerName="extract" Jun 08 22:56:55.433569 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.433519 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerName="extract" Jun 08 22:56:55.433569 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.433537 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerName="util" Jun 08 22:56:55.433569 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.433542 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerName="util" Jun 08 22:56:55.433569 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.433550 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerName="pull" Jun 08 22:56:55.433569 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.433555 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerName="pull" Jun 08 22:56:55.433710 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.433599 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d122723-638d-46fd-ae09-d6cb6763ad4f" containerName="extract" Jun 08 22:56:55.436852 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.436836 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.438992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.438956 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 08 22:56:55.438992 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.438978 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7hrnz\"" Jun 08 22:56:55.439170 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.438996 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 08 22:56:55.439170 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.438997 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 08 22:56:55.439170 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.439061 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 08 22:56:55.448808 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.448783 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz"] Jun 08 22:56:55.544991 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.544956 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx6sg\" (UniqueName: \"kubernetes.io/projected/49ece150-ed54-4dd0-9783-298906f765e6-kube-api-access-nx6sg\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.545381 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.545024 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49ece150-ed54-4dd0-9783-298906f765e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.545381 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.545049 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49ece150-ed54-4dd0-9783-298906f765e6-manifests\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.545381 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.545068 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49ece150-ed54-4dd0-9783-298906f765e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.545381 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.545109 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49ece150-ed54-4dd0-9783-298906f765e6-tmp\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.547615 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.547591 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:55.646165 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.646077 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx6sg\" (UniqueName: \"kubernetes.io/projected/49ece150-ed54-4dd0-9783-298906f765e6-kube-api-access-nx6sg\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.646165 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.646121 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49ece150-ed54-4dd0-9783-298906f765e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.646165 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.646142 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49ece150-ed54-4dd0-9783-298906f765e6-manifests\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.646165 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.646162 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49ece150-ed54-4dd0-9783-298906f765e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.646527 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.646233 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49ece150-ed54-4dd0-9783-298906f765e6-tmp\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.646616 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.646593 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/49ece150-ed54-4dd0-9783-298906f765e6-manifests\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.648792 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.648763 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49ece150-ed54-4dd0-9783-298906f765e6-tmp\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.648907 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.648798 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:55.648907 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.648854 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49ece150-ed54-4dd0-9783-298906f765e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.648907 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.648881 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/49ece150-ed54-4dd0-9783-298906f765e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.655680 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.655653 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx6sg\" (UniqueName: \"kubernetes.io/projected/49ece150-ed54-4dd0-9783-298906f765e6-kube-api-access-nx6sg\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-f64tz\" (UID: \"49ece150-ed54-4dd0-9783-298906f765e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.746703 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.746667 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:56:55.748841 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.748805 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:55.850106 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.850078 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:55.876038 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.876016 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz"] Jun 08 22:56:55.878800 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:56:55.878774 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ece150_ed54_4dd0_9783_298906f765e6.slice/crio-7e21f3cf01ce7afb441734728849047697b9e579269bbd311cf6af1149e397c3 WatchSource:0}: Error finding container 7e21f3cf01ce7afb441734728849047697b9e579269bbd311cf6af1149e397c3: Status 404 returned error can't find the container with id 7e21f3cf01ce7afb441734728849047697b9e579269bbd311cf6af1149e397c3 Jun 08 22:56:55.915148 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.915073 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:55.977428 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:55.977382 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:56.099848 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:56.099820 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:56:56.450962 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:56.450924 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" event={"ID":"49ece150-ed54-4dd0-9783-298906f765e6","Type":"ContainerStarted","Data":"7e21f3cf01ce7afb441734728849047697b9e579269bbd311cf6af1149e397c3"} Jun 08 22:56:59.468579 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:56:59.468540 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" event={"ID":"49ece150-ed54-4dd0-9783-298906f765e6","Type":"ContainerStarted","Data":"8d661d6bcad9839ce561b480c98917bebfdcf5ca02c4cae238f8b35dba62ad21"} Jun 08 22:57:00.473582 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:00.473544 2567 generic.go:356] "Generic (PLEG): container finished" podID="49ece150-ed54-4dd0-9783-298906f765e6" containerID="8d661d6bcad9839ce561b480c98917bebfdcf5ca02c4cae238f8b35dba62ad21" exitCode=0 Jun 08 22:57:00.473984 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:00.473637 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" event={"ID":"49ece150-ed54-4dd0-9783-298906f765e6","Type":"ContainerDied","Data":"8d661d6bcad9839ce561b480c98917bebfdcf5ca02c4cae238f8b35dba62ad21"} Jun 08 22:57:01.479121 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:01.479081 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" event={"ID":"49ece150-ed54-4dd0-9783-298906f765e6","Type":"ContainerStarted","Data":"ea0c628de2586a051fe40dad4052fcdf716b3248ce120dbc38eec99ffda35eca"} Jun 08 22:57:01.479595 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:01.479211 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:57:01.498546 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:01.498496 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" podStartSLOduration=2.299194204 podStartE2EDuration="6.498479385s" totalImagesPullingTime="3.281832525s" totalInitContainerRuntime="917.452656ms" isStatefulPod=false podCreationTimestamp="2026-06-08 22:56:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:01.497067959 +0000 UTC m=+543.305679814" watchObservedRunningTime="2026-06-08 22:57:01.498479385 +0000 UTC m=+543.307091238" Jun 08 22:57:12.486289 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:12.486257 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-f64tz" Jun 08 22:57:12.991524 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:12.991487 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:13.065788 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.065756 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:13.066144 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.066124 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn"] Jun 08 22:57:13.069363 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.069347 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.071242 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.071222 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 08 22:57:13.071332 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.071266 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 08 22:57:13.071560 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.071541 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ngwhf\"" Jun 08 22:57:13.071677 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.071566 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 08 22:57:13.071746 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.071702 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 08 22:57:13.072654 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.072634 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 08 22:57:13.079729 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.079710 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn"] Jun 08 22:57:13.165559 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.165521 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-manager-config\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.165559 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.165564 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-metrics-cert\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.165798 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.165640 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-cert\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.165798 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.165679 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnhp4\" (UniqueName: \"kubernetes.io/projected/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-kube-api-access-lnhp4\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.266373 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.266298 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-cert\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.266373 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.266338 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnhp4\" (UniqueName: \"kubernetes.io/projected/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-kube-api-access-lnhp4\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.266373 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.266377 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-manager-config\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.266667 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.266415 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-metrics-cert\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.267155 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.267131 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-manager-config\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.268851 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.268827 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-cert\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.268955 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.268868 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-metrics-cert\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.280629 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.280605 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnhp4\" (UniqueName: \"kubernetes.io/projected/d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44-kube-api-access-lnhp4\") pod \"lws-controller-manager-667ddfcf76-r2gxn\" (UID: \"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44\") " pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.378909 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.378876 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:13.505757 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.505729 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn"] Jun 08 22:57:13.507998 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:57:13.507971 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e5bd47_1a9d_4727_aae3_3ed1d7fcee44.slice/crio-f7e8a2fc9bbb8d95c94db4e964d7c3d236e36fac647d90dfb63b4bdc0df6a581 WatchSource:0}: Error finding container f7e8a2fc9bbb8d95c94db4e964d7c3d236e36fac647d90dfb63b4bdc0df6a581: Status 404 returned error can't find the container with id f7e8a2fc9bbb8d95c94db4e964d7c3d236e36fac647d90dfb63b4bdc0df6a581 Jun 08 22:57:13.528382 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.528305 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" event={"ID":"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44","Type":"ContainerStarted","Data":"f7e8a2fc9bbb8d95c94db4e964d7c3d236e36fac647d90dfb63b4bdc0df6a581"} Jun 08 22:57:13.991977 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:13.991939 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:14.097896 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.097864 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:14.258089 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.258011 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:14.383835 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.383796 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:14.504258 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.502782 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk"] Jun 08 22:57:14.507278 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.507243 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.509357 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.509297 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7qzv5\"" Jun 08 22:57:14.510207 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.510185 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 08 22:57:14.510321 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.510213 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 08 22:57:14.519350 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.519312 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk"] Jun 08 22:57:14.576706 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.576673 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.576872 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.576725 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.576872 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.576797 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4675n\" (UniqueName: \"kubernetes.io/projected/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-kube-api-access-4675n\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.649096 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.649050 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:14.677280 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.677236 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4675n\" (UniqueName: \"kubernetes.io/projected/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-kube-api-access-4675n\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.677470 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.677328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.677470 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.677374 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.677872 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.677848 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.677957 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.677883 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.690981 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.690951 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4675n\" (UniqueName: \"kubernetes.io/projected/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-kube-api-access-4675n\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:14.819798 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:14.819716 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:15.146200 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.146154 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:15.251305 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.251281 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:15.273199 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.273174 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk"] Jun 08 22:57:15.274997 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:57:15.274971 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19f2ecc6_b443_4cdf_8cde_ca6b8974baee.slice/crio-d1276811c166d4927af052d8463094c07046b9e7873750a11413abab7ec6e86f WatchSource:0}: Error finding container d1276811c166d4927af052d8463094c07046b9e7873750a11413abab7ec6e86f: Status 404 returned error can't find the container with id d1276811c166d4927af052d8463094c07046b9e7873750a11413abab7ec6e86f Jun 08 22:57:15.352687 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.352645 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:15.445956 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.445927 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:15.539814 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.539778 2567 generic.go:356] "Generic (PLEG): container finished" podID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerID="b36a2130240e5d3eae113f7d5889686b3f8b2175cd8c1a4b4e4a43b8ec6268f7" exitCode=0 Jun 08 22:57:15.540271 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.539869 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" event={"ID":"19f2ecc6-b443-4cdf-8cde-ca6b8974baee","Type":"ContainerDied","Data":"b36a2130240e5d3eae113f7d5889686b3f8b2175cd8c1a4b4e4a43b8ec6268f7"} Jun 08 22:57:15.540271 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.539971 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" event={"ID":"19f2ecc6-b443-4cdf-8cde-ca6b8974baee","Type":"ContainerStarted","Data":"d1276811c166d4927af052d8463094c07046b9e7873750a11413abab7ec6e86f"} Jun 08 22:57:15.541784 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.541756 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" event={"ID":"d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44","Type":"ContainerStarted","Data":"5ccaaf8a229ed1a2d0dd961f155d5d5a05c2b32a4c79c6296ae172a227f2b8c2"} Jun 08 22:57:15.541923 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.541906 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:15.580793 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.580743 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" podStartSLOduration=0.891792367 podStartE2EDuration="2.580728047s" totalImagesPullingTime="1.68893568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:15.579586175 +0000 UTC m=+557.388198029" watchObservedRunningTime="2026-06-08 22:57:15.580728047 +0000 UTC m=+557.389339901" Jun 08 22:57:15.592121 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:15.592094 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:16.530353 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:16.530277 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:16.547328 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:16.547301 2567 generic.go:356] "Generic (PLEG): container finished" podID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerID="e37db80caeecbdb9b8563ce35ad6d17ceabefdda86962ea3ca313d55f152722e" exitCode=0 Jun 08 22:57:16.547683 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:16.547405 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" event={"ID":"19f2ecc6-b443-4cdf-8cde-ca6b8974baee","Type":"ContainerDied","Data":"e37db80caeecbdb9b8563ce35ad6d17ceabefdda86962ea3ca313d55f152722e"} Jun 08 22:57:17.554506 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:17.554473 2567 generic.go:356] "Generic (PLEG): container finished" podID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerID="08af572c5c5b82be8a3580c49371898768b9712e560b3f115ef2acca8e6d13f7" exitCode=0 Jun 08 22:57:17.554857 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:17.554535 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" event={"ID":"19f2ecc6-b443-4cdf-8cde-ca6b8974baee","Type":"ContainerDied","Data":"08af572c5c5b82be8a3580c49371898768b9712e560b3f115ef2acca8e6d13f7"} Jun 08 22:57:18.682648 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.682626 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:18.709609 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.709580 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-util\") pod \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " Jun 08 22:57:18.709754 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.709627 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4675n\" (UniqueName: \"kubernetes.io/projected/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-kube-api-access-4675n\") pod \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " Jun 08 22:57:18.709754 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.709676 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-bundle\") pod \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\" (UID: \"19f2ecc6-b443-4cdf-8cde-ca6b8974baee\") " Jun 08 22:57:18.710641 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.710613 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-bundle" (OuterVolumeSpecName: "bundle") pod "19f2ecc6-b443-4cdf-8cde-ca6b8974baee" (UID: "19f2ecc6-b443-4cdf-8cde-ca6b8974baee"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:57:18.712070 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.712035 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-kube-api-access-4675n" (OuterVolumeSpecName: "kube-api-access-4675n") pod "19f2ecc6-b443-4cdf-8cde-ca6b8974baee" (UID: "19f2ecc6-b443-4cdf-8cde-ca6b8974baee"). InnerVolumeSpecName "kube-api-access-4675n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:57:18.718043 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.718014 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-util" (OuterVolumeSpecName: "util") pod "19f2ecc6-b443-4cdf-8cde-ca6b8974baee" (UID: "19f2ecc6-b443-4cdf-8cde-ca6b8974baee"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:57:18.810488 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.810381 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:18.810488 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.810433 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4675n\" (UniqueName: \"kubernetes.io/projected/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-kube-api-access-4675n\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:18.810488 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:18.810444 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19f2ecc6-b443-4cdf-8cde-ca6b8974baee-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:19.564416 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:19.564363 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" event={"ID":"19f2ecc6-b443-4cdf-8cde-ca6b8974baee","Type":"ContainerDied","Data":"d1276811c166d4927af052d8463094c07046b9e7873750a11413abab7ec6e86f"} Jun 08 22:57:19.564566 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:19.564464 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1276811c166d4927af052d8463094c07046b9e7873750a11413abab7ec6e86f" Jun 08 22:57:19.564566 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:19.564374 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vj2lk" Jun 08 22:57:22.398132 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:22.398085 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:22.498592 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:22.498562 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:23.404244 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.404211 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz"] Jun 08 22:57:23.404635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.404485 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerName="pull" Jun 08 22:57:23.404635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.404496 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerName="pull" Jun 08 22:57:23.404635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.404510 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerName="util" Jun 08 22:57:23.404635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.404516 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerName="util" Jun 08 22:57:23.404635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.404529 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerName="extract" Jun 08 22:57:23.404635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.404534 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerName="extract" Jun 08 22:57:23.404635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.404588 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="19f2ecc6-b443-4cdf-8cde-ca6b8974baee" containerName="extract" Jun 08 22:57:23.407836 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.407819 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.411971 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.411943 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 08 22:57:23.412745 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.412728 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 08 22:57:23.414435 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.414415 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7qzv5\"" Jun 08 22:57:23.435036 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.435000 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz"] Jun 08 22:57:23.449568 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.449544 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.449706 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.449579 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.449706 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.449599 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9ps7\" (UniqueName: \"kubernetes.io/projected/6852342c-e132-4a4c-b230-96268e81958d-kube-api-access-p9ps7\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.550758 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.550724 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.550758 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.550757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9ps7\" (UniqueName: \"kubernetes.io/projected/6852342c-e132-4a4c-b230-96268e81958d-kube-api-access-p9ps7\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.550967 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.550816 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.551123 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.551101 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.551163 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.551126 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.571353 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.571321 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9ps7\" (UniqueName: \"kubernetes.io/projected/6852342c-e132-4a4c-b230-96268e81958d-kube-api-access-p9ps7\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.678869 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.678838 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:23.716781 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.716752 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:23.780331 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.780224 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:23.852749 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:23.852726 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz"] Jun 08 22:57:23.854305 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:57:23.854280 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6852342c_e132_4a4c_b230_96268e81958d.slice/crio-b9b176b725a9bbdaf48a5a6d8dde29bc1b87bbd21651c8c533c1f6bb6d2af41e WatchSource:0}: Error finding container b9b176b725a9bbdaf48a5a6d8dde29bc1b87bbd21651c8c533c1f6bb6d2af41e: Status 404 returned error can't find the container with id b9b176b725a9bbdaf48a5a6d8dde29bc1b87bbd21651c8c533c1f6bb6d2af41e Jun 08 22:57:24.043886 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:24.043856 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:24.152317 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:24.152286 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:24.585939 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:24.585906 2567 generic.go:356] "Generic (PLEG): container finished" podID="6852342c-e132-4a4c-b230-96268e81958d" containerID="8b724ffbf34d009171f42ee1b526377731d24727a68d32231d94015e4fb07150" exitCode=0 Jun 08 22:57:24.586292 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:24.585991 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" event={"ID":"6852342c-e132-4a4c-b230-96268e81958d","Type":"ContainerDied","Data":"8b724ffbf34d009171f42ee1b526377731d24727a68d32231d94015e4fb07150"} Jun 08 22:57:24.586292 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:24.586053 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" event={"ID":"6852342c-e132-4a4c-b230-96268e81958d","Type":"ContainerStarted","Data":"b9b176b725a9bbdaf48a5a6d8dde29bc1b87bbd21651c8c533c1f6bb6d2af41e"} Jun 08 22:57:24.677844 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:24.677812 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:25.580928 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:25.580897 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:25.592706 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:25.592625 2567 generic.go:356] "Generic (PLEG): container finished" podID="6852342c-e132-4a4c-b230-96268e81958d" containerID="44775d6cbbe59370f0e1c36d6af318b50e46e9c95e416a12e604a91e3f384310" exitCode=0 Jun 08 22:57:25.593045 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:25.592702 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" event={"ID":"6852342c-e132-4a4c-b230-96268e81958d","Type":"ContainerDied","Data":"44775d6cbbe59370f0e1c36d6af318b50e46e9c95e416a12e604a91e3f384310"} Jun 08 22:57:25.699602 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:25.699571 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:25.791532 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:25.791504 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:26.143756 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:26.143675 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:26.549549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:26.549521 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-667ddfcf76-r2gxn" Jun 08 22:57:26.599178 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:26.599149 2567 generic.go:356] "Generic (PLEG): container finished" podID="6852342c-e132-4a4c-b230-96268e81958d" containerID="0b9baf90c13ba713b46c5de407bfed9586d73a595eda52f7d2893ff293481e44" exitCode=0 Jun 08 22:57:26.599569 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:26.599227 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" event={"ID":"6852342c-e132-4a4c-b230-96268e81958d","Type":"ContainerDied","Data":"0b9baf90c13ba713b46c5de407bfed9586d73a595eda52f7d2893ff293481e44"} Jun 08 22:57:26.901211 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:26.901126 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:26.997455 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:26.997424 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:27.628004 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.627960 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-6976cdc864-z5nsf"] Jun 08 22:57:27.634235 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.634209 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" Jun 08 22:57:27.637415 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.637378 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zsvn4\"" Jun 08 22:57:27.637534 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.637413 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 08 22:57:27.649423 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.649386 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6976cdc864-z5nsf"] Jun 08 22:57:27.681969 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.681934 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csp4s\" (UniqueName: \"kubernetes.io/projected/2d164051-cdfc-45c0-a62d-174d093d5e87-kube-api-access-csp4s\") pod \"dashboard-redirect-6976cdc864-z5nsf\" (UID: \"2d164051-cdfc-45c0-a62d-174d093d5e87\") " pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" Jun 08 22:57:27.682093 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.682055 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/2d164051-cdfc-45c0-a62d-174d093d5e87-redirect-config\") pod \"dashboard-redirect-6976cdc864-z5nsf\" (UID: \"2d164051-cdfc-45c0-a62d-174d093d5e87\") " pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" Jun 08 22:57:27.685251 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.685230 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:27.734927 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.734904 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:27.782872 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.782844 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9ps7\" (UniqueName: \"kubernetes.io/projected/6852342c-e132-4a4c-b230-96268e81958d-kube-api-access-p9ps7\") pod \"6852342c-e132-4a4c-b230-96268e81958d\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " Jun 08 22:57:27.783029 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.782918 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-util\") pod \"6852342c-e132-4a4c-b230-96268e81958d\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " Jun 08 22:57:27.783029 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.782958 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-bundle\") pod \"6852342c-e132-4a4c-b230-96268e81958d\" (UID: \"6852342c-e132-4a4c-b230-96268e81958d\") " Jun 08 22:57:27.783113 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.783069 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/2d164051-cdfc-45c0-a62d-174d093d5e87-redirect-config\") pod \"dashboard-redirect-6976cdc864-z5nsf\" (UID: \"2d164051-cdfc-45c0-a62d-174d093d5e87\") " pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" Jun 08 22:57:27.783162 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.783113 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csp4s\" (UniqueName: \"kubernetes.io/projected/2d164051-cdfc-45c0-a62d-174d093d5e87-kube-api-access-csp4s\") pod \"dashboard-redirect-6976cdc864-z5nsf\" (UID: \"2d164051-cdfc-45c0-a62d-174d093d5e87\") " pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" Jun 08 22:57:27.783811 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.783786 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/2d164051-cdfc-45c0-a62d-174d093d5e87-redirect-config\") pod \"dashboard-redirect-6976cdc864-z5nsf\" (UID: \"2d164051-cdfc-45c0-a62d-174d093d5e87\") " pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" Jun 08 22:57:27.784113 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.784087 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-bundle" (OuterVolumeSpecName: "bundle") pod "6852342c-e132-4a4c-b230-96268e81958d" (UID: "6852342c-e132-4a4c-b230-96268e81958d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:57:27.785086 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.785058 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6852342c-e132-4a4c-b230-96268e81958d-kube-api-access-p9ps7" (OuterVolumeSpecName: "kube-api-access-p9ps7") pod "6852342c-e132-4a4c-b230-96268e81958d" (UID: "6852342c-e132-4a4c-b230-96268e81958d"). InnerVolumeSpecName "kube-api-access-p9ps7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:57:27.788902 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.788869 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-util" (OuterVolumeSpecName: "util") pod "6852342c-e132-4a4c-b230-96268e81958d" (UID: "6852342c-e132-4a4c-b230-96268e81958d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:57:27.791226 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.791206 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csp4s\" (UniqueName: \"kubernetes.io/projected/2d164051-cdfc-45c0-a62d-174d093d5e87-kube-api-access-csp4s\") pod \"dashboard-redirect-6976cdc864-z5nsf\" (UID: \"2d164051-cdfc-45c0-a62d-174d093d5e87\") " pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" Jun 08 22:57:27.884463 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.884354 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:27.884463 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.884383 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9ps7\" (UniqueName: \"kubernetes.io/projected/6852342c-e132-4a4c-b230-96268e81958d-kube-api-access-p9ps7\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:27.884463 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.884414 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6852342c-e132-4a4c-b230-96268e81958d-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:27.946027 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:27.945993 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" Jun 08 22:57:28.074750 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:28.074697 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6976cdc864-z5nsf"] Jun 08 22:57:28.077192 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:57:28.077167 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d164051_cdfc_45c0_a62d_174d093d5e87.slice/crio-4dd5796fab83c022c84ad54de6609ccdbbf849d73b7502c5795a256e341c24ef WatchSource:0}: Error finding container 4dd5796fab83c022c84ad54de6609ccdbbf849d73b7502c5795a256e341c24ef: Status 404 returned error can't find the container with id 4dd5796fab83c022c84ad54de6609ccdbbf849d73b7502c5795a256e341c24ef Jun 08 22:57:28.609570 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:28.609533 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" event={"ID":"2d164051-cdfc-45c0-a62d-174d093d5e87","Type":"ContainerStarted","Data":"4dd5796fab83c022c84ad54de6609ccdbbf849d73b7502c5795a256e341c24ef"} Jun 08 22:57:28.619224 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:28.619197 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" Jun 08 22:57:28.622469 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:28.622432 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2rvbz" event={"ID":"6852342c-e132-4a4c-b230-96268e81958d","Type":"ContainerDied","Data":"b9b176b725a9bbdaf48a5a6d8dde29bc1b87bbd21651c8c533c1f6bb6d2af41e"} Jun 08 22:57:28.622640 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:28.622478 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9b176b725a9bbdaf48a5a6d8dde29bc1b87bbd21651c8c533c1f6bb6d2af41e" Jun 08 22:57:31.633192 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:31.633156 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" event={"ID":"2d164051-cdfc-45c0-a62d-174d093d5e87","Type":"ContainerStarted","Data":"b9c6954527a7c5ff4efca0340b9b8ef57fbb25546882f5b57a55f9e914299d49"} Jun 08 22:57:31.651611 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:31.651565 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-6976cdc864-z5nsf" podStartSLOduration=1.563192135 podStartE2EDuration="4.65154819s" totalImagesPullingTime="3.088356055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:31.650293667 +0000 UTC m=+573.458905522" watchObservedRunningTime="2026-06-08 22:57:31.65154819 +0000 UTC m=+573.460160055" Jun 08 22:57:35.217498 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:35.217463 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:35.318594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:35.318532 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:37.114875 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:37.114765 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:37.167095 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:37.167059 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:37.608007 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:37.607972 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:42.663802 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.663763 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:42.732056 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.732026 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:42.908487 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.908453 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng"] Jun 08 22:57:42.908725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.908713 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6852342c-e132-4a4c-b230-96268e81958d" containerName="pull" Jun 08 22:57:42.908772 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.908727 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6852342c-e132-4a4c-b230-96268e81958d" containerName="pull" Jun 08 22:57:42.908772 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.908740 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6852342c-e132-4a4c-b230-96268e81958d" containerName="util" Jun 08 22:57:42.908772 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.908746 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6852342c-e132-4a4c-b230-96268e81958d" containerName="util" Jun 08 22:57:42.908772 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.908755 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6852342c-e132-4a4c-b230-96268e81958d" containerName="extract" Jun 08 22:57:42.908772 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.908761 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6852342c-e132-4a4c-b230-96268e81958d" containerName="extract" Jun 08 22:57:42.908915 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.908805 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6852342c-e132-4a4c-b230-96268e81958d" containerName="extract" Jun 08 22:57:42.911611 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.911595 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:42.913696 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.913663 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 08 22:57:42.914083 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.913733 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 08 22:57:42.914083 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.913764 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 08 22:57:42.914083 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.913754 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-llvd6\"" Jun 08 22:57:42.924999 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.924976 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng"] Jun 08 22:57:42.938330 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:42.938305 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:43.007195 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.007156 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.007195 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.007194 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp8zj\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-kube-api-access-jp8zj\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.007451 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.007246 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.007451 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.007267 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.007451 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.007282 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.007451 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.007298 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec94a406-d579-4299-9f1e-c503eeff996b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.007451 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.007367 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.007617 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.007454 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.007617 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.007472 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec94a406-d579-4299-9f1e-c503eeff996b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.037315 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.037289 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:43.108592 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.108545 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.108592 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.108602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.108819 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.108621 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.108819 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.108718 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec94a406-d579-4299-9f1e-c503eeff996b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.108819 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.108755 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.108819 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.108813 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.109023 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.108838 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec94a406-d579-4299-9f1e-c503eeff996b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.109023 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.108870 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.109023 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.108895 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp8zj\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-kube-api-access-jp8zj\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.109180 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.109047 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.109180 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.109161 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.109267 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.109202 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.109373 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.109351 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.109637 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.109621 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec94a406-d579-4299-9f1e-c503eeff996b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.110921 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.110902 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.111174 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.111158 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec94a406-d579-4299-9f1e-c503eeff996b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.127186 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.127161 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.127612 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.127593 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp8zj\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-kube-api-access-jp8zj\") pod \"data-science-gateway-data-science-gateway-class-5c5946766f2q5ng\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.221089 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.221047 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:43.272577 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.272509 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q"] Jun 08 22:57:43.276252 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.276224 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.298428 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.298383 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q"] Jun 08 22:57:43.314981 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.314928 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:43.357127 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.357099 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng"] Jun 08 22:57:43.359386 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:57:43.359362 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec94a406_d579_4299_9f1e_c503eeff996b.slice/crio-d544875d24a72ad1b5a0f9e875320a641e101b1779608f07bbe6e0c7dab49123 WatchSource:0}: Error finding container d544875d24a72ad1b5a0f9e875320a641e101b1779608f07bbe6e0c7dab49123: Status 404 returned error can't find the container with id d544875d24a72ad1b5a0f9e875320a641e101b1779608f07bbe6e0c7dab49123 Jun 08 22:57:43.411721 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.411687 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.411721 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.411719 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vzh8\" (UniqueName: \"kubernetes.io/projected/2c8258ce-933b-45be-92fd-803b1bc74503-kube-api-access-8vzh8\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.411957 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.411745 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.411957 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.411850 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c8258ce-933b-45be-92fd-803b1bc74503-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.411957 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.411899 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.411957 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.411945 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.412160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.412002 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c8258ce-933b-45be-92fd-803b1bc74503-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.412160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.412049 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.412160 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.412080 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c8258ce-933b-45be-92fd-803b1bc74503-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.512837 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.512742 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.512837 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.512788 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c8258ce-933b-45be-92fd-803b1bc74503-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513062 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.512915 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513062 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.512959 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c8258ce-933b-45be-92fd-803b1bc74503-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513062 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513189 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513060 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vzh8\" (UniqueName: \"kubernetes.io/projected/2c8258ce-933b-45be-92fd-803b1bc74503-kube-api-access-8vzh8\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513189 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513104 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513189 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513152 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c8258ce-933b-45be-92fd-803b1bc74503-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513345 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513197 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513345 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513257 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513489 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513471 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513574 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513552 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513617 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513606 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.513708 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.513693 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c8258ce-933b-45be-92fd-803b1bc74503-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.515127 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.515104 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c8258ce-933b-45be-92fd-803b1bc74503-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.515698 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.515682 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c8258ce-933b-45be-92fd-803b1bc74503-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.521039 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.521020 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c8258ce-933b-45be-92fd-803b1bc74503-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.521551 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.521533 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vzh8\" (UniqueName: \"kubernetes.io/projected/2c8258ce-933b-45be-92fd-803b1bc74503-kube-api-access-8vzh8\") pod \"data-science-gateway-data-science-gateway-class-5b556588bczhx6q\" (UID: \"2c8258ce-933b-45be-92fd-803b1bc74503\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.590146 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.590106 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:43.685065 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.685034 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" event={"ID":"ec94a406-d579-4299-9f1e-c503eeff996b","Type":"ContainerStarted","Data":"d544875d24a72ad1b5a0f9e875320a641e101b1779608f07bbe6e0c7dab49123"} Jun 08 22:57:43.723022 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:43.722988 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q"] Jun 08 22:57:43.723678 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:57:43.723656 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8258ce_933b_45be_92fd_803b1bc74503.slice/crio-ac86d53011300d282ae46b4264c255f1d3b1a89feaa927c2b71d9cdfc91ad42b WatchSource:0}: Error finding container ac86d53011300d282ae46b4264c255f1d3b1a89feaa927c2b71d9cdfc91ad42b: Status 404 returned error can't find the container with id ac86d53011300d282ae46b4264c255f1d3b1a89feaa927c2b71d9cdfc91ad42b Jun 08 22:57:44.692572 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:44.692533 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" event={"ID":"2c8258ce-933b-45be-92fd-803b1bc74503","Type":"ContainerStarted","Data":"ac86d53011300d282ae46b4264c255f1d3b1a89feaa927c2b71d9cdfc91ad42b"} Jun 08 22:57:48.421543 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:48.421505 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 22:57:48.421771 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:48.421589 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 22:57:48.421771 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:48.421622 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 22:57:48.422896 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:48.422872 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:57:48.711560 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:48.711476 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" event={"ID":"2c8258ce-933b-45be-92fd-803b1bc74503","Type":"ContainerStarted","Data":"011d9d1efa6c4aaffff3be25bb32a46f8728617c2e83956c3482eb354b64ee94"} Jun 08 22:57:48.728555 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:48.728503 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" podStartSLOduration=1.03308255 podStartE2EDuration="5.728485753s" totalImagesPullingTime="4.695403203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:48.726882898 +0000 UTC m=+590.535494752" watchObservedRunningTime="2026-06-08 22:57:48.728485753 +0000 UTC m=+590.537097607" Jun 08 22:57:49.591003 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:49.590968 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:49.595592 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:49.595570 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:49.716253 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:49.716224 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:49.717223 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:49.717207 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bczhx6q" Jun 08 22:57:49.761808 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:49.761776 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng"] Jun 08 22:57:50.573520 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:50.573477 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 22:57:50.573640 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:50.573557 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 22:57:50.573640 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:50.573588 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 22:57:50.722961 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:50.722917 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" event={"ID":"ec94a406-d579-4299-9f1e-c503eeff996b","Type":"ContainerStarted","Data":"f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919"} Jun 08 22:57:50.723412 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:50.722964 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" podUID="ec94a406-d579-4299-9f1e-c503eeff996b" containerName="istio-proxy" containerID="cri-o://f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919" gracePeriod=30 Jun 08 22:57:50.740865 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:50.740812 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" podStartSLOduration=1.529007116 podStartE2EDuration="8.740794584s" totalImagesPullingTime="7.211787468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:50.739518317 +0000 UTC m=+592.548130187" watchObservedRunningTime="2026-06-08 22:57:50.740794584 +0000 UTC m=+592.549406437" Jun 08 22:57:55.960833 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:55.960812 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:56.120856 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.120767 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec94a406-d579-4299-9f1e-c503eeff996b-istiod-ca-cert\") pod \"ec94a406-d579-4299-9f1e-c503eeff996b\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " Jun 08 22:57:56.120856 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.120809 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-certs\") pod \"ec94a406-d579-4299-9f1e-c503eeff996b\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " Jun 08 22:57:56.120856 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.120832 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-credential-socket\") pod \"ec94a406-d579-4299-9f1e-c503eeff996b\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " Jun 08 22:57:56.121112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.120862 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-socket\") pod \"ec94a406-d579-4299-9f1e-c503eeff996b\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " Jun 08 22:57:56.121112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.120963 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-data\") pod \"ec94a406-d579-4299-9f1e-c503eeff996b\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " Jun 08 22:57:56.121112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121016 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-istio-token\") pod \"ec94a406-d579-4299-9f1e-c503eeff996b\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " Jun 08 22:57:56.121112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121072 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp8zj\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-kube-api-access-jp8zj\") pod \"ec94a406-d579-4299-9f1e-c503eeff996b\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " Jun 08 22:57:56.121112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121094 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec94a406-d579-4299-9f1e-c503eeff996b" (UID: "ec94a406-d579-4299-9f1e-c503eeff996b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:57:56.121112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121109 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec94a406-d579-4299-9f1e-c503eeff996b-istio-podinfo\") pod \"ec94a406-d579-4299-9f1e-c503eeff996b\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " Jun 08 22:57:56.121424 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121138 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec94a406-d579-4299-9f1e-c503eeff996b" (UID: "ec94a406-d579-4299-9f1e-c503eeff996b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:57:56.121424 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121165 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-envoy\") pod \"ec94a406-d579-4299-9f1e-c503eeff996b\" (UID: \"ec94a406-d579-4299-9f1e-c503eeff996b\") " Jun 08 22:57:56.121424 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121191 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec94a406-d579-4299-9f1e-c503eeff996b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec94a406-d579-4299-9f1e-c503eeff996b" (UID: "ec94a406-d579-4299-9f1e-c503eeff996b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:57:56.121424 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121289 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec94a406-d579-4299-9f1e-c503eeff996b" (UID: "ec94a406-d579-4299-9f1e-c503eeff996b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:57:56.121424 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121374 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec94a406-d579-4299-9f1e-c503eeff996b" (UID: "ec94a406-d579-4299-9f1e-c503eeff996b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:57:56.121424 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121427 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec94a406-d579-4299-9f1e-c503eeff996b-istiod-ca-cert\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:56.121722 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121439 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-certs\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:56.121722 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121450 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-credential-socket\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:56.121722 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.121465 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-workload-socket\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:56.123444 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.123418 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec94a406-d579-4299-9f1e-c503eeff996b" (UID: "ec94a406-d579-4299-9f1e-c503eeff996b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:57:56.123593 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.123571 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-kube-api-access-jp8zj" (OuterVolumeSpecName: "kube-api-access-jp8zj") pod "ec94a406-d579-4299-9f1e-c503eeff996b" (UID: "ec94a406-d579-4299-9f1e-c503eeff996b"). InnerVolumeSpecName "kube-api-access-jp8zj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:57:56.123667 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.123593 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec94a406-d579-4299-9f1e-c503eeff996b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec94a406-d579-4299-9f1e-c503eeff996b" (UID: "ec94a406-d579-4299-9f1e-c503eeff996b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 08 22:57:56.123667 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.123587 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec94a406-d579-4299-9f1e-c503eeff996b" (UID: "ec94a406-d579-4299-9f1e-c503eeff996b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:57:56.222632 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.222600 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-data\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:56.222632 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.222629 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-istio-token\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:56.222632 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.222641 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp8zj\" (UniqueName: \"kubernetes.io/projected/ec94a406-d579-4299-9f1e-c503eeff996b-kube-api-access-jp8zj\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:56.222850 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.222651 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec94a406-d579-4299-9f1e-c503eeff996b-istio-podinfo\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:56.222850 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.222660 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec94a406-d579-4299-9f1e-c503eeff996b-istio-envoy\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:57:56.749747 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.749710 2567 generic.go:356] "Generic (PLEG): container finished" podID="ec94a406-d579-4299-9f1e-c503eeff996b" containerID="f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919" exitCode=0 Jun 08 22:57:56.750006 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.749751 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" Jun 08 22:57:56.750006 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.749804 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" event={"ID":"ec94a406-d579-4299-9f1e-c503eeff996b","Type":"ContainerDied","Data":"f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919"} Jun 08 22:57:56.750006 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.749850 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng" event={"ID":"ec94a406-d579-4299-9f1e-c503eeff996b","Type":"ContainerDied","Data":"d544875d24a72ad1b5a0f9e875320a641e101b1779608f07bbe6e0c7dab49123"} Jun 08 22:57:56.750006 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.749870 2567 scope.go:117] "RemoveContainer" containerID="f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919" Jun 08 22:57:56.763002 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.762981 2567 scope.go:117] "RemoveContainer" containerID="f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919" Jun 08 22:57:56.763278 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:57:56.763257 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919\": container with ID starting with f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919 not found: ID does not exist" containerID="f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919" Jun 08 22:57:56.763325 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.763287 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919"} err="failed to get container status \"f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919\": rpc error: code = NotFound desc = could not find container \"f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919\": container with ID starting with f27dc06c85b9921daff28e516a8ba9c460a0e303aa012c96be584f935aff0919 not found: ID does not exist" Jun 08 22:57:56.768698 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.768675 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng"] Jun 08 22:57:56.773678 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:56.773654 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5c5946766f2q5ng"] Jun 08 22:57:58.597415 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:58.597370 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 22:57:58.598831 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:58.598806 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 22:57:58.602172 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:58.602149 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 22:57:58.603843 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:58.603826 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 22:57:58.620412 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:57:58.620370 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec94a406-d579-4299-9f1e-c503eeff996b" path="/var/lib/kubelet/pods/ec94a406-d579-4299-9f1e-c503eeff996b/volumes" Jun 08 22:58:14.142089 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:14.141991 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:14.246807 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:14.246775 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:15.501814 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.501776 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:15.601754 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.601724 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:15.637506 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.637474 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kqzb6"] Jun 08 22:58:15.637741 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.637730 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec94a406-d579-4299-9f1e-c503eeff996b" containerName="istio-proxy" Jun 08 22:58:15.637793 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.637742 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec94a406-d579-4299-9f1e-c503eeff996b" containerName="istio-proxy" Jun 08 22:58:15.637828 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.637796 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec94a406-d579-4299-9f1e-c503eeff996b" containerName="istio-proxy" Jun 08 22:58:15.640528 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.640510 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" Jun 08 22:58:15.642328 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.642306 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 08 22:58:15.642815 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.642795 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rsvp9\"" Jun 08 22:58:15.642815 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.642805 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 08 22:58:15.650077 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.650048 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kqzb6"] Jun 08 22:58:15.676946 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.676914 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48nn4\" (UniqueName: \"kubernetes.io/projected/78ed9daa-c8f4-42d7-b375-49c1d80918a1-kube-api-access-48nn4\") pod \"kuadrant-operator-catalog-kqzb6\" (UID: \"78ed9daa-c8f4-42d7-b375-49c1d80918a1\") " pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" Jun 08 22:58:15.699951 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.699927 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:15.777987 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.777907 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48nn4\" (UniqueName: \"kubernetes.io/projected/78ed9daa-c8f4-42d7-b375-49c1d80918a1-kube-api-access-48nn4\") pod \"kuadrant-operator-catalog-kqzb6\" (UID: \"78ed9daa-c8f4-42d7-b375-49c1d80918a1\") " pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" Jun 08 22:58:15.788854 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.788823 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48nn4\" (UniqueName: \"kubernetes.io/projected/78ed9daa-c8f4-42d7-b375-49c1d80918a1-kube-api-access-48nn4\") pod \"kuadrant-operator-catalog-kqzb6\" (UID: \"78ed9daa-c8f4-42d7-b375-49c1d80918a1\") " pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" Jun 08 22:58:15.951598 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:15.951560 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" Jun 08 22:58:16.074948 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.074928 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kqzb6"] Jun 08 22:58:16.077152 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:58:16.077117 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ed9daa_c8f4_42d7_b375_49c1d80918a1.slice/crio-185fd19ea5ed728c0567d2bb24b540aad4c49d8bfbd1e42fe80b631de1dacc36 WatchSource:0}: Error finding container 185fd19ea5ed728c0567d2bb24b540aad4c49d8bfbd1e42fe80b631de1dacc36: Status 404 returned error can't find the container with id 185fd19ea5ed728c0567d2bb24b540aad4c49d8bfbd1e42fe80b631de1dacc36 Jun 08 22:58:16.237056 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.237020 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kqzb6"] Jun 08 22:58:16.252211 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.252181 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dw466"] Jun 08 22:58:16.255984 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.255968 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dw466" Jun 08 22:58:16.262055 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.262032 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dw466"] Jun 08 22:58:16.281686 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.281620 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvd88\" (UniqueName: \"kubernetes.io/projected/ffab74d5-d160-4e39-83d5-898a91ea4d35-kube-api-access-gvd88\") pod \"kuadrant-operator-catalog-dw466\" (UID: \"ffab74d5-d160-4e39-83d5-898a91ea4d35\") " pod="kuadrant-system/kuadrant-operator-catalog-dw466" Jun 08 22:58:16.382446 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.382414 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvd88\" (UniqueName: \"kubernetes.io/projected/ffab74d5-d160-4e39-83d5-898a91ea4d35-kube-api-access-gvd88\") pod \"kuadrant-operator-catalog-dw466\" (UID: \"ffab74d5-d160-4e39-83d5-898a91ea4d35\") " pod="kuadrant-system/kuadrant-operator-catalog-dw466" Jun 08 22:58:16.389858 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.389823 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvd88\" (UniqueName: \"kubernetes.io/projected/ffab74d5-d160-4e39-83d5-898a91ea4d35-kube-api-access-gvd88\") pod \"kuadrant-operator-catalog-dw466\" (UID: \"ffab74d5-d160-4e39-83d5-898a91ea4d35\") " pod="kuadrant-system/kuadrant-operator-catalog-dw466" Jun 08 22:58:16.566613 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.566537 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dw466" Jun 08 22:58:16.695978 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.695954 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dw466"] Jun 08 22:58:16.706738 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:58:16.706711 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffab74d5_d160_4e39_83d5_898a91ea4d35.slice/crio-a5751ee9cb9b84a7bd086bde6ce52d94e337af59fc101e08e8f7713ff78de43d WatchSource:0}: Error finding container a5751ee9cb9b84a7bd086bde6ce52d94e337af59fc101e08e8f7713ff78de43d: Status 404 returned error can't find the container with id a5751ee9cb9b84a7bd086bde6ce52d94e337af59fc101e08e8f7713ff78de43d Jun 08 22:58:16.838480 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.838382 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dw466" event={"ID":"ffab74d5-d160-4e39-83d5-898a91ea4d35","Type":"ContainerStarted","Data":"a5751ee9cb9b84a7bd086bde6ce52d94e337af59fc101e08e8f7713ff78de43d"} Jun 08 22:58:16.839411 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:16.839363 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" event={"ID":"78ed9daa-c8f4-42d7-b375-49c1d80918a1","Type":"ContainerStarted","Data":"185fd19ea5ed728c0567d2bb24b540aad4c49d8bfbd1e42fe80b631de1dacc36"} Jun 08 22:58:18.849621 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:18.849577 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" event={"ID":"78ed9daa-c8f4-42d7-b375-49c1d80918a1","Type":"ContainerStarted","Data":"ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23"} Jun 08 22:58:18.849621 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:18.849613 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" podUID="78ed9daa-c8f4-42d7-b375-49c1d80918a1" containerName="registry-server" containerID="cri-o://ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23" gracePeriod=2 Jun 08 22:58:18.851050 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:18.851020 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dw466" event={"ID":"ffab74d5-d160-4e39-83d5-898a91ea4d35","Type":"ContainerStarted","Data":"9132c771bb1ba89b2cd93195a2c8e058d018ecc33edf96dbcf4431a188c22d89"} Jun 08 22:58:18.863735 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:18.863688 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" podStartSLOduration=1.746775257 podStartE2EDuration="3.863673838s" totalImagesPullingTime="2.116898581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:58:18.861829324 +0000 UTC m=+620.670441179" watchObservedRunningTime="2026-06-08 22:58:18.863673838 +0000 UTC m=+620.672285692" Jun 08 22:58:18.873715 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:18.873675 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dw466" podStartSLOduration=1.294769491 podStartE2EDuration="2.873662804s" totalImagesPullingTime="1.578893313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:58:18.873499815 +0000 UTC m=+620.682111672" watchObservedRunningTime="2026-06-08 22:58:18.873662804 +0000 UTC m=+620.682274657" Jun 08 22:58:19.093365 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.093342 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" Jun 08 22:58:19.204462 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.204434 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48nn4\" (UniqueName: \"kubernetes.io/projected/78ed9daa-c8f4-42d7-b375-49c1d80918a1-kube-api-access-48nn4\") pod \"78ed9daa-c8f4-42d7-b375-49c1d80918a1\" (UID: \"78ed9daa-c8f4-42d7-b375-49c1d80918a1\") " Jun 08 22:58:19.206606 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.206582 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ed9daa-c8f4-42d7-b375-49c1d80918a1-kube-api-access-48nn4" (OuterVolumeSpecName: "kube-api-access-48nn4") pod "78ed9daa-c8f4-42d7-b375-49c1d80918a1" (UID: "78ed9daa-c8f4-42d7-b375-49c1d80918a1"). InnerVolumeSpecName "kube-api-access-48nn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:58:19.305146 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.305116 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48nn4\" (UniqueName: \"kubernetes.io/projected/78ed9daa-c8f4-42d7-b375-49c1d80918a1-kube-api-access-48nn4\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:19.857535 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.857505 2567 generic.go:356] "Generic (PLEG): container finished" podID="78ed9daa-c8f4-42d7-b375-49c1d80918a1" containerID="ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23" exitCode=0 Jun 08 22:58:19.857950 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.857550 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" Jun 08 22:58:19.857950 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.857582 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" event={"ID":"78ed9daa-c8f4-42d7-b375-49c1d80918a1","Type":"ContainerDied","Data":"ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23"} Jun 08 22:58:19.857950 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.857621 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kqzb6" event={"ID":"78ed9daa-c8f4-42d7-b375-49c1d80918a1","Type":"ContainerDied","Data":"185fd19ea5ed728c0567d2bb24b540aad4c49d8bfbd1e42fe80b631de1dacc36"} Jun 08 22:58:19.857950 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.857641 2567 scope.go:117] "RemoveContainer" containerID="ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23" Jun 08 22:58:19.867127 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.867107 2567 scope.go:117] "RemoveContainer" containerID="ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23" Jun 08 22:58:19.867369 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:58:19.867354 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23\": container with ID starting with ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23 not found: ID does not exist" containerID="ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23" Jun 08 22:58:19.867438 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.867378 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23"} err="failed to get container status \"ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23\": rpc error: code = NotFound desc = could not find container \"ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23\": container with ID starting with ae529a67e30b03a644023eea9e25678a976d316253e5a16971d6375a997e6b23 not found: ID does not exist" Jun 08 22:58:19.876701 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.876678 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kqzb6"] Jun 08 22:58:19.881899 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:19.881878 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kqzb6"] Jun 08 22:58:20.620882 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:20.620842 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78ed9daa-c8f4-42d7-b375-49c1d80918a1" path="/var/lib/kubelet/pods/78ed9daa-c8f4-42d7-b375-49c1d80918a1/volumes" Jun 08 22:58:26.567384 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:26.567337 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-dw466" Jun 08 22:58:26.567838 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:26.567433 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-dw466" Jun 08 22:58:26.589069 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:26.589047 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-dw466" Jun 08 22:58:26.909884 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:26.909817 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-dw466" Jun 08 22:58:30.974183 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:30.974150 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l"] Jun 08 22:58:30.974646 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:30.974442 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ed9daa-c8f4-42d7-b375-49c1d80918a1" containerName="registry-server" Jun 08 22:58:30.974646 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:30.974454 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ed9daa-c8f4-42d7-b375-49c1d80918a1" containerName="registry-server" Jun 08 22:58:30.974646 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:30.974500 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="78ed9daa-c8f4-42d7-b375-49c1d80918a1" containerName="registry-server" Jun 08 22:58:30.981111 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:30.981091 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:30.983172 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:30.983148 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ksx8k\"" Jun 08 22:58:30.984489 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:30.984465 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l"] Jun 08 22:58:31.103112 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.103072 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.103286 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.103166 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl5jw\" (UniqueName: \"kubernetes.io/projected/8686956c-bf4f-41c7-ba79-82abc5b759b6-kube-api-access-cl5jw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.103286 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.103209 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.204455 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.204420 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl5jw\" (UniqueName: \"kubernetes.io/projected/8686956c-bf4f-41c7-ba79-82abc5b759b6-kube-api-access-cl5jw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.204589 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.204463 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.204589 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.204511 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.204877 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.204856 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.204941 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.204923 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.211472 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.211448 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl5jw\" (UniqueName: \"kubernetes.io/projected/8686956c-bf4f-41c7-ba79-82abc5b759b6-kube-api-access-cl5jw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.291756 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.291676 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:31.423699 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.423497 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l"] Jun 08 22:58:31.425960 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:58:31.425928 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8686956c_bf4f_41c7_ba79_82abc5b759b6.slice/crio-9794b6cf44e1318b90599da77c67583688c929802ca820eff1b11ac48f2191df WatchSource:0}: Error finding container 9794b6cf44e1318b90599da77c67583688c929802ca820eff1b11ac48f2191df: Status 404 returned error can't find the container with id 9794b6cf44e1318b90599da77c67583688c929802ca820eff1b11ac48f2191df Jun 08 22:58:31.574285 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.574208 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r"] Jun 08 22:58:31.577446 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.577429 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.585780 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.585754 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r"] Jun 08 22:58:31.708338 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.708299 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dhdx\" (UniqueName: \"kubernetes.io/projected/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-kube-api-access-6dhdx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.708551 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.708359 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.708551 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.708483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.809124 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.809083 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dhdx\" (UniqueName: \"kubernetes.io/projected/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-kube-api-access-6dhdx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.809321 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.809148 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.809321 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.809199 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.809629 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.809604 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.809704 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.809615 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.816798 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.816780 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dhdx\" (UniqueName: \"kubernetes.io/projected/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-kube-api-access-6dhdx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.886826 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.886752 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:31.912356 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.912325 2567 generic.go:356] "Generic (PLEG): container finished" podID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerID="64db4246d483811f112b9f5be047e5c8f7ebdebb3d29dc676f282649b8a211fa" exitCode=0 Jun 08 22:58:31.912522 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.912414 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" event={"ID":"8686956c-bf4f-41c7-ba79-82abc5b759b6","Type":"ContainerDied","Data":"64db4246d483811f112b9f5be047e5c8f7ebdebb3d29dc676f282649b8a211fa"} Jun 08 22:58:31.912522 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:31.912477 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" event={"ID":"8686956c-bf4f-41c7-ba79-82abc5b759b6","Type":"ContainerStarted","Data":"9794b6cf44e1318b90599da77c67583688c929802ca820eff1b11ac48f2191df"} Jun 08 22:58:32.014241 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.014213 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r"] Jun 08 22:58:32.016316 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:58:32.016285 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8edcc6_a901_4a51_93bf_bc25c1eef43a.slice/crio-e7c83b02b9de7cf89c12ff5c7b7c00cbe0552535962964c6f1c07deee9468ba5 WatchSource:0}: Error finding container e7c83b02b9de7cf89c12ff5c7b7c00cbe0552535962964c6f1c07deee9468ba5: Status 404 returned error can't find the container with id e7c83b02b9de7cf89c12ff5c7b7c00cbe0552535962964c6f1c07deee9468ba5 Jun 08 22:58:32.175493 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.175449 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp"] Jun 08 22:58:32.178819 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.178803 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.184929 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.184902 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp"] Jun 08 22:58:32.314799 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.314756 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fbw9\" (UniqueName: \"kubernetes.io/projected/076ea65d-8964-4234-9fa5-8f4801f8b658-kube-api-access-2fbw9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.314970 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.314826 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.314970 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.314859 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.415598 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.415551 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.415789 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.415606 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.415789 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.415684 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fbw9\" (UniqueName: \"kubernetes.io/projected/076ea65d-8964-4234-9fa5-8f4801f8b658-kube-api-access-2fbw9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.415967 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.415944 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.416036 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.415965 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.422573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.422546 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fbw9\" (UniqueName: \"kubernetes.io/projected/076ea65d-8964-4234-9fa5-8f4801f8b658-kube-api-access-2fbw9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.488793 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.488729 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:32.573539 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.573502 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb"] Jun 08 22:58:32.577172 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.577149 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.588843 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.588777 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb"] Jun 08 22:58:32.617190 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.617160 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5jwn\" (UniqueName: \"kubernetes.io/projected/836e726b-c5b2-40d9-9673-bee64e44e278-kube-api-access-m5jwn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.617190 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.617195 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.617413 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.617225 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.631997 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.631943 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp"] Jun 08 22:58:32.643078 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:58:32.643050 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076ea65d_8964_4234_9fa5_8f4801f8b658.slice/crio-504b0dddd5775e770fdf69956f211ffd2fd8a1483688a8d87c39ab6cd962fdd7 WatchSource:0}: Error finding container 504b0dddd5775e770fdf69956f211ffd2fd8a1483688a8d87c39ab6cd962fdd7: Status 404 returned error can't find the container with id 504b0dddd5775e770fdf69956f211ffd2fd8a1483688a8d87c39ab6cd962fdd7 Jun 08 22:58:32.718600 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.718569 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5jwn\" (UniqueName: \"kubernetes.io/projected/836e726b-c5b2-40d9-9673-bee64e44e278-kube-api-access-m5jwn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.718740 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.718613 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.718740 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.718672 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.719044 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.719019 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.719129 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.719049 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.727348 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.727321 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5jwn\" (UniqueName: \"kubernetes.io/projected/836e726b-c5b2-40d9-9673-bee64e44e278-kube-api-access-m5jwn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.893035 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.892998 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:32.919054 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.919024 2567 generic.go:356] "Generic (PLEG): container finished" podID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerID="d332aec0220b1dc715546641b11c7b131fa2f79f70f08fe18329f3ab8a7c34d7" exitCode=0 Jun 08 22:58:32.919191 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.919094 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" event={"ID":"8686956c-bf4f-41c7-ba79-82abc5b759b6","Type":"ContainerDied","Data":"d332aec0220b1dc715546641b11c7b131fa2f79f70f08fe18329f3ab8a7c34d7"} Jun 08 22:58:32.920841 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.920813 2567 generic.go:356] "Generic (PLEG): container finished" podID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerID="0b8faa75024a49dd7b7fcfcb4322b7d3e79685a7ee087d2152b7bf1c8b0291bd" exitCode=0 Jun 08 22:58:32.920965 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.920942 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" event={"ID":"076ea65d-8964-4234-9fa5-8f4801f8b658","Type":"ContainerDied","Data":"0b8faa75024a49dd7b7fcfcb4322b7d3e79685a7ee087d2152b7bf1c8b0291bd"} Jun 08 22:58:32.921097 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.920981 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" event={"ID":"076ea65d-8964-4234-9fa5-8f4801f8b658","Type":"ContainerStarted","Data":"504b0dddd5775e770fdf69956f211ffd2fd8a1483688a8d87c39ab6cd962fdd7"} Jun 08 22:58:32.922826 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.922806 2567 generic.go:356] "Generic (PLEG): container finished" podID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerID="e2cd1c03598757db1159e6b1b96534f0737dcd9a41f7eea96a7fcc9e10de5fd4" exitCode=0 Jun 08 22:58:32.922923 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.922844 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" event={"ID":"9f8edcc6-a901-4a51-93bf-bc25c1eef43a","Type":"ContainerDied","Data":"e2cd1c03598757db1159e6b1b96534f0737dcd9a41f7eea96a7fcc9e10de5fd4"} Jun 08 22:58:32.922923 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:32.922874 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" event={"ID":"9f8edcc6-a901-4a51-93bf-bc25c1eef43a","Type":"ContainerStarted","Data":"e7c83b02b9de7cf89c12ff5c7b7c00cbe0552535962964c6f1c07deee9468ba5"} Jun 08 22:58:33.029495 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.029422 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb"] Jun 08 22:58:33.031592 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:58:33.031564 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod836e726b_c5b2_40d9_9673_bee64e44e278.slice/crio-a8e4ea2ad4d8e59c735713aa542446c3f500ed0ff2afe2821506e8907bebd616 WatchSource:0}: Error finding container a8e4ea2ad4d8e59c735713aa542446c3f500ed0ff2afe2821506e8907bebd616: Status 404 returned error can't find the container with id a8e4ea2ad4d8e59c735713aa542446c3f500ed0ff2afe2821506e8907bebd616 Jun 08 22:58:33.928646 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.928610 2567 generic.go:356] "Generic (PLEG): container finished" podID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerID="2ddc82819d48b6381584bb97f362d20455e44003523cee8704d3b07fd43b4c01" exitCode=0 Jun 08 22:58:33.928818 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.928709 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" event={"ID":"9f8edcc6-a901-4a51-93bf-bc25c1eef43a","Type":"ContainerDied","Data":"2ddc82819d48b6381584bb97f362d20455e44003523cee8704d3b07fd43b4c01"} Jun 08 22:58:33.931179 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.931160 2567 generic.go:356] "Generic (PLEG): container finished" podID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerID="ff8ec4a18dd10dcfc1f024e9aa5ccbcf7dea7a72167052c745448253ae156cb0" exitCode=0 Jun 08 22:58:33.931279 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.931193 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" event={"ID":"8686956c-bf4f-41c7-ba79-82abc5b759b6","Type":"ContainerDied","Data":"ff8ec4a18dd10dcfc1f024e9aa5ccbcf7dea7a72167052c745448253ae156cb0"} Jun 08 22:58:33.932961 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.932943 2567 generic.go:356] "Generic (PLEG): container finished" podID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerID="dcd8256f347e038a986de04a90df2d3f7e4de34eae82ae0cdd86da9ce4cc4f3b" exitCode=0 Jun 08 22:58:33.933091 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.933020 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" event={"ID":"076ea65d-8964-4234-9fa5-8f4801f8b658","Type":"ContainerDied","Data":"dcd8256f347e038a986de04a90df2d3f7e4de34eae82ae0cdd86da9ce4cc4f3b"} Jun 08 22:58:33.934594 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.934574 2567 generic.go:356] "Generic (PLEG): container finished" podID="836e726b-c5b2-40d9-9673-bee64e44e278" containerID="9130baa84a1c79acfc5b02d3a997bebda7808cb7b87b7976bd30d2366c71930e" exitCode=0 Jun 08 22:58:33.934673 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.934620 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" event={"ID":"836e726b-c5b2-40d9-9673-bee64e44e278","Type":"ContainerDied","Data":"9130baa84a1c79acfc5b02d3a997bebda7808cb7b87b7976bd30d2366c71930e"} Jun 08 22:58:33.934739 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:33.934662 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" event={"ID":"836e726b-c5b2-40d9-9673-bee64e44e278","Type":"ContainerStarted","Data":"a8e4ea2ad4d8e59c735713aa542446c3f500ed0ff2afe2821506e8907bebd616"} Jun 08 22:58:34.941581 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:34.941545 2567 generic.go:356] "Generic (PLEG): container finished" podID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerID="cd1f5310f482d12592daf1c73b4ed08a9067bf7d192234a626b0789659d9ca3b" exitCode=0 Jun 08 22:58:34.942005 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:34.941627 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" event={"ID":"076ea65d-8964-4234-9fa5-8f4801f8b658","Type":"ContainerDied","Data":"cd1f5310f482d12592daf1c73b4ed08a9067bf7d192234a626b0789659d9ca3b"} Jun 08 22:58:34.943355 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:34.943331 2567 generic.go:356] "Generic (PLEG): container finished" podID="836e726b-c5b2-40d9-9673-bee64e44e278" containerID="1d3270ff172fef823e1433426e7b3336f06b76e9b17937b99ee0d1281b85aff7" exitCode=0 Jun 08 22:58:34.943490 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:34.943422 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" event={"ID":"836e726b-c5b2-40d9-9673-bee64e44e278","Type":"ContainerDied","Data":"1d3270ff172fef823e1433426e7b3336f06b76e9b17937b99ee0d1281b85aff7"} Jun 08 22:58:34.945632 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:34.945610 2567 generic.go:356] "Generic (PLEG): container finished" podID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerID="a6adb50e96accb6d0403725a4cac3b0ef16bfda2eb604b0efd9621007a1f182e" exitCode=0 Jun 08 22:58:34.945743 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:34.945694 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" event={"ID":"9f8edcc6-a901-4a51-93bf-bc25c1eef43a","Type":"ContainerDied","Data":"a6adb50e96accb6d0403725a4cac3b0ef16bfda2eb604b0efd9621007a1f182e"} Jun 08 22:58:35.074149 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.074119 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:35.139825 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.139791 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl5jw\" (UniqueName: \"kubernetes.io/projected/8686956c-bf4f-41c7-ba79-82abc5b759b6-kube-api-access-cl5jw\") pod \"8686956c-bf4f-41c7-ba79-82abc5b759b6\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " Jun 08 22:58:35.139970 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.139833 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-util\") pod \"8686956c-bf4f-41c7-ba79-82abc5b759b6\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " Jun 08 22:58:35.139970 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.139876 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-bundle\") pod \"8686956c-bf4f-41c7-ba79-82abc5b759b6\" (UID: \"8686956c-bf4f-41c7-ba79-82abc5b759b6\") " Jun 08 22:58:35.140412 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.140363 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-bundle" (OuterVolumeSpecName: "bundle") pod "8686956c-bf4f-41c7-ba79-82abc5b759b6" (UID: "8686956c-bf4f-41c7-ba79-82abc5b759b6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:58:35.141775 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.141751 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8686956c-bf4f-41c7-ba79-82abc5b759b6-kube-api-access-cl5jw" (OuterVolumeSpecName: "kube-api-access-cl5jw") pod "8686956c-bf4f-41c7-ba79-82abc5b759b6" (UID: "8686956c-bf4f-41c7-ba79-82abc5b759b6"). InnerVolumeSpecName "kube-api-access-cl5jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:58:35.145291 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.145273 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-util" (OuterVolumeSpecName: "util") pod "8686956c-bf4f-41c7-ba79-82abc5b759b6" (UID: "8686956c-bf4f-41c7-ba79-82abc5b759b6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:58:35.240735 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.240701 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl5jw\" (UniqueName: \"kubernetes.io/projected/8686956c-bf4f-41c7-ba79-82abc5b759b6-kube-api-access-cl5jw\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:35.240735 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.240731 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:35.240735 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.240742 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8686956c-bf4f-41c7-ba79-82abc5b759b6-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:35.952261 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.952232 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" Jun 08 22:58:35.952701 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.952233 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l" event={"ID":"8686956c-bf4f-41c7-ba79-82abc5b759b6","Type":"ContainerDied","Data":"9794b6cf44e1318b90599da77c67583688c929802ca820eff1b11ac48f2191df"} Jun 08 22:58:35.952701 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.952354 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9794b6cf44e1318b90599da77c67583688c929802ca820eff1b11ac48f2191df" Jun 08 22:58:35.954451 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.954420 2567 generic.go:356] "Generic (PLEG): container finished" podID="836e726b-c5b2-40d9-9673-bee64e44e278" containerID="e8f46f243827179be0bd554b77b98f188358d98495b37f828bb3fc97eb6aedcb" exitCode=0 Jun 08 22:58:35.954587 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:35.954484 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" event={"ID":"836e726b-c5b2-40d9-9673-bee64e44e278","Type":"ContainerDied","Data":"e8f46f243827179be0bd554b77b98f188358d98495b37f828bb3fc97eb6aedcb"} Jun 08 22:58:36.130000 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.129972 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:36.131979 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.131962 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:36.248505 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.248368 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-bundle\") pod \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " Jun 08 22:58:36.248505 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.248465 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-util\") pod \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " Jun 08 22:58:36.248746 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.248534 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-util\") pod \"076ea65d-8964-4234-9fa5-8f4801f8b658\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " Jun 08 22:58:36.248746 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.248566 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fbw9\" (UniqueName: \"kubernetes.io/projected/076ea65d-8964-4234-9fa5-8f4801f8b658-kube-api-access-2fbw9\") pod \"076ea65d-8964-4234-9fa5-8f4801f8b658\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " Jun 08 22:58:36.248746 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.248598 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-bundle\") pod \"076ea65d-8964-4234-9fa5-8f4801f8b658\" (UID: \"076ea65d-8964-4234-9fa5-8f4801f8b658\") " Jun 08 22:58:36.248746 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.248626 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dhdx\" (UniqueName: \"kubernetes.io/projected/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-kube-api-access-6dhdx\") pod \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\" (UID: \"9f8edcc6-a901-4a51-93bf-bc25c1eef43a\") " Jun 08 22:58:36.249024 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.248999 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-bundle" (OuterVolumeSpecName: "bundle") pod "9f8edcc6-a901-4a51-93bf-bc25c1eef43a" (UID: "9f8edcc6-a901-4a51-93bf-bc25c1eef43a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:58:36.249167 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.249135 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-bundle" (OuterVolumeSpecName: "bundle") pod "076ea65d-8964-4234-9fa5-8f4801f8b658" (UID: "076ea65d-8964-4234-9fa5-8f4801f8b658"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:58:36.250954 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.250922 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-kube-api-access-6dhdx" (OuterVolumeSpecName: "kube-api-access-6dhdx") pod "9f8edcc6-a901-4a51-93bf-bc25c1eef43a" (UID: "9f8edcc6-a901-4a51-93bf-bc25c1eef43a"). InnerVolumeSpecName "kube-api-access-6dhdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:58:36.251253 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.251234 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/076ea65d-8964-4234-9fa5-8f4801f8b658-kube-api-access-2fbw9" (OuterVolumeSpecName: "kube-api-access-2fbw9") pod "076ea65d-8964-4234-9fa5-8f4801f8b658" (UID: "076ea65d-8964-4234-9fa5-8f4801f8b658"). InnerVolumeSpecName "kube-api-access-2fbw9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:58:36.254530 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.254509 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-util" (OuterVolumeSpecName: "util") pod "9f8edcc6-a901-4a51-93bf-bc25c1eef43a" (UID: "9f8edcc6-a901-4a51-93bf-bc25c1eef43a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:58:36.255126 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.255105 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-util" (OuterVolumeSpecName: "util") pod "076ea65d-8964-4234-9fa5-8f4801f8b658" (UID: "076ea65d-8964-4234-9fa5-8f4801f8b658"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:58:36.349435 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.349369 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:36.349435 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.349429 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:36.349435 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.349438 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:36.349667 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.349447 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fbw9\" (UniqueName: \"kubernetes.io/projected/076ea65d-8964-4234-9fa5-8f4801f8b658-kube-api-access-2fbw9\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:36.349667 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.349457 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076ea65d-8964-4234-9fa5-8f4801f8b658-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:36.349667 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.349466 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dhdx\" (UniqueName: \"kubernetes.io/projected/9f8edcc6-a901-4a51-93bf-bc25c1eef43a-kube-api-access-6dhdx\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:36.960639 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.960613 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" Jun 08 22:58:36.961045 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.960634 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp" event={"ID":"076ea65d-8964-4234-9fa5-8f4801f8b658","Type":"ContainerDied","Data":"504b0dddd5775e770fdf69956f211ffd2fd8a1483688a8d87c39ab6cd962fdd7"} Jun 08 22:58:36.961045 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.960687 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="504b0dddd5775e770fdf69956f211ffd2fd8a1483688a8d87c39ab6cd962fdd7" Jun 08 22:58:36.962710 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.962689 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" Jun 08 22:58:36.962851 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.962728 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r" event={"ID":"9f8edcc6-a901-4a51-93bf-bc25c1eef43a","Type":"ContainerDied","Data":"e7c83b02b9de7cf89c12ff5c7b7c00cbe0552535962964c6f1c07deee9468ba5"} Jun 08 22:58:36.962851 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:36.962762 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7c83b02b9de7cf89c12ff5c7b7c00cbe0552535962964c6f1c07deee9468ba5" Jun 08 22:58:37.088882 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.088855 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:37.157040 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.157003 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-bundle\") pod \"836e726b-c5b2-40d9-9673-bee64e44e278\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " Jun 08 22:58:37.157242 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.157066 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-util\") pod \"836e726b-c5b2-40d9-9673-bee64e44e278\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " Jun 08 22:58:37.157242 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.157091 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5jwn\" (UniqueName: \"kubernetes.io/projected/836e726b-c5b2-40d9-9673-bee64e44e278-kube-api-access-m5jwn\") pod \"836e726b-c5b2-40d9-9673-bee64e44e278\" (UID: \"836e726b-c5b2-40d9-9673-bee64e44e278\") " Jun 08 22:58:37.157680 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.157652 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-bundle" (OuterVolumeSpecName: "bundle") pod "836e726b-c5b2-40d9-9673-bee64e44e278" (UID: "836e726b-c5b2-40d9-9673-bee64e44e278"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:58:37.159233 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.159205 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/836e726b-c5b2-40d9-9673-bee64e44e278-kube-api-access-m5jwn" (OuterVolumeSpecName: "kube-api-access-m5jwn") pod "836e726b-c5b2-40d9-9673-bee64e44e278" (UID: "836e726b-c5b2-40d9-9673-bee64e44e278"). InnerVolumeSpecName "kube-api-access-m5jwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:58:37.162780 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.162758 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-util" (OuterVolumeSpecName: "util") pod "836e726b-c5b2-40d9-9673-bee64e44e278" (UID: "836e726b-c5b2-40d9-9673-bee64e44e278"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:58:37.258446 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.258336 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:37.258446 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.258367 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5jwn\" (UniqueName: \"kubernetes.io/projected/836e726b-c5b2-40d9-9673-bee64e44e278-kube-api-access-m5jwn\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:37.258446 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.258378 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/836e726b-c5b2-40d9-9673-bee64e44e278-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:58:37.969281 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.969253 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" Jun 08 22:58:37.969281 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.969273 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb" event={"ID":"836e726b-c5b2-40d9-9673-bee64e44e278","Type":"ContainerDied","Data":"a8e4ea2ad4d8e59c735713aa542446c3f500ed0ff2afe2821506e8907bebd616"} Jun 08 22:58:37.969740 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:37.969318 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8e4ea2ad4d8e59c735713aa542446c3f500ed0ff2afe2821506e8907bebd616" Jun 08 22:58:44.476557 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:44.476524 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:47.100166 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100133 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq"] Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100422 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerName="pull" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100433 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerName="pull" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100444 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerName="util" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100449 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerName="util" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100458 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerName="extract" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100464 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerName="extract" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100472 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerName="util" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100477 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerName="util" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100482 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerName="pull" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100486 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerName="pull" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100491 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerName="pull" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100496 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerName="pull" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100501 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerName="extract" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100508 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerName="extract" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100516 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerName="extract" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100521 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerName="extract" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100527 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="836e726b-c5b2-40d9-9673-bee64e44e278" containerName="extract" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100531 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="836e726b-c5b2-40d9-9673-bee64e44e278" containerName="extract" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100539 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerName="util" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100544 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerName="util" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100553 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="836e726b-c5b2-40d9-9673-bee64e44e278" containerName="util" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100558 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="836e726b-c5b2-40d9-9673-bee64e44e278" containerName="util" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100565 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="836e726b-c5b2-40d9-9673-bee64e44e278" containerName="pull" Jun 08 22:58:47.100573 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100570 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="836e726b-c5b2-40d9-9673-bee64e44e278" containerName="pull" Jun 08 22:58:47.101356 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100612 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="836e726b-c5b2-40d9-9673-bee64e44e278" containerName="extract" Jun 08 22:58:47.101356 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100618 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8edcc6-a901-4a51-93bf-bc25c1eef43a" containerName="extract" Jun 08 22:58:47.101356 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100626 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8686956c-bf4f-41c7-ba79-82abc5b759b6" containerName="extract" Jun 08 22:58:47.101356 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.100633 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="076ea65d-8964-4234-9fa5-8f4801f8b658" containerName="extract" Jun 08 22:58:47.103405 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.103380 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" Jun 08 22:58:47.105414 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.105376 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 08 22:58:47.105516 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.105481 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-d4sxz\"" Jun 08 22:58:47.118317 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.118285 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq"] Jun 08 22:58:47.246593 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.246548 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb8sw\" (UniqueName: \"kubernetes.io/projected/17596fba-079f-449e-86da-e8b9b5bf5512-kube-api-access-nb8sw\") pod \"dns-operator-controller-manager-864dd78c4c-k4kmq\" (UID: \"17596fba-079f-449e-86da-e8b9b5bf5512\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" Jun 08 22:58:47.347003 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.346967 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb8sw\" (UniqueName: \"kubernetes.io/projected/17596fba-079f-449e-86da-e8b9b5bf5512-kube-api-access-nb8sw\") pod \"dns-operator-controller-manager-864dd78c4c-k4kmq\" (UID: \"17596fba-079f-449e-86da-e8b9b5bf5512\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" Jun 08 22:58:47.357374 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.357318 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb8sw\" (UniqueName: \"kubernetes.io/projected/17596fba-079f-449e-86da-e8b9b5bf5512-kube-api-access-nb8sw\") pod \"dns-operator-controller-manager-864dd78c4c-k4kmq\" (UID: \"17596fba-079f-449e-86da-e8b9b5bf5512\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" Jun 08 22:58:47.415294 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.415259 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" Jun 08 22:58:47.543536 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.543511 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq"] Jun 08 22:58:47.545030 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:58:47.545004 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17596fba_079f_449e_86da_e8b9b5bf5512.slice/crio-5a31c8fed9ccc3bf3f7e20c6e99a7b56daf79ed023716c3002c39399a8f26d38 WatchSource:0}: Error finding container 5a31c8fed9ccc3bf3f7e20c6e99a7b56daf79ed023716c3002c39399a8f26d38: Status 404 returned error can't find the container with id 5a31c8fed9ccc3bf3f7e20c6e99a7b56daf79ed023716c3002c39399a8f26d38 Jun 08 22:58:47.547114 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:47.547099 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 22:58:48.017469 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:48.017430 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" event={"ID":"17596fba-079f-449e-86da-e8b9b5bf5512","Type":"ContainerStarted","Data":"5a31c8fed9ccc3bf3f7e20c6e99a7b56daf79ed023716c3002c39399a8f26d38"} Jun 08 22:58:48.664154 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:48.664117 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:50.028695 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.028653 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" event={"ID":"17596fba-079f-449e-86da-e8b9b5bf5512","Type":"ContainerStarted","Data":"d97a119bc6a7ed272e47df016253c12ff5a66cdb49a78838cfe10d1a9f60e95c"} Jun 08 22:58:50.029173 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.028757 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" Jun 08 22:58:50.045665 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.045617 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" podStartSLOduration=0.784716727 podStartE2EDuration="3.045603109s" totalImagesPullingTime="2.260886382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:58:50.043042068 +0000 UTC m=+651.851653935" watchObservedRunningTime="2026-06-08 22:58:50.045603109 +0000 UTC m=+651.854214962" Jun 08 22:58:50.559679 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.559642 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8"] Jun 08 22:58:50.562973 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.562952 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:58:50.564712 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.564688 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jdb6w\"" Jun 08 22:58:50.573081 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.573058 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8"] Jun 08 22:58:50.675649 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.675614 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5m9m\" (UniqueName: \"kubernetes.io/projected/8b7ec8ab-243a-4177-862f-5749bb45a8e1-kube-api-access-h5m9m\") pod \"limitador-operator-controller-manager-859d7b78bd-2lxp8\" (UID: \"8b7ec8ab-243a-4177-862f-5749bb45a8e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:58:50.776450 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.776409 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5m9m\" (UniqueName: \"kubernetes.io/projected/8b7ec8ab-243a-4177-862f-5749bb45a8e1-kube-api-access-h5m9m\") pod \"limitador-operator-controller-manager-859d7b78bd-2lxp8\" (UID: \"8b7ec8ab-243a-4177-862f-5749bb45a8e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:58:50.791306 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.791274 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5m9m\" (UniqueName: \"kubernetes.io/projected/8b7ec8ab-243a-4177-862f-5749bb45a8e1-kube-api-access-h5m9m\") pod \"limitador-operator-controller-manager-859d7b78bd-2lxp8\" (UID: \"8b7ec8ab-243a-4177-862f-5749bb45a8e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:58:50.873987 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:50.873898 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:58:51.002162 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:51.002136 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8"] Jun 08 22:58:51.004091 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:58:51.004057 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7ec8ab_243a_4177_862f_5749bb45a8e1.slice/crio-1d058793305bff41859f3667437131d7226fa25a80d962652a4156fa99cff021 WatchSource:0}: Error finding container 1d058793305bff41859f3667437131d7226fa25a80d962652a4156fa99cff021: Status 404 returned error can't find the container with id 1d058793305bff41859f3667437131d7226fa25a80d962652a4156fa99cff021 Jun 08 22:58:51.035747 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:51.035708 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" event={"ID":"8b7ec8ab-243a-4177-862f-5749bb45a8e1","Type":"ContainerStarted","Data":"1d058793305bff41859f3667437131d7226fa25a80d962652a4156fa99cff021"} Jun 08 22:58:51.081338 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:51.081309 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:51.182544 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:51.182513 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:51.300436 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:51.300380 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:51.659892 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:51.659806 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:53.047748 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:53.047714 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" event={"ID":"8b7ec8ab-243a-4177-862f-5749bb45a8e1","Type":"ContainerStarted","Data":"b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad"} Jun 08 22:58:53.048118 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:53.047789 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:58:53.063251 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:53.063206 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" podStartSLOduration=1.469083884 podStartE2EDuration="3.063190504s" totalImagesPullingTime="1.59410662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:58:53.060813041 +0000 UTC m=+654.869424894" watchObservedRunningTime="2026-06-08 22:58:53.063190504 +0000 UTC m=+654.871802420" Jun 08 22:58:53.946588 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:53.946529 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:55.116957 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:55.116923 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:55.914597 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:55.914558 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:56.013448 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:56.013415 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:56.119724 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:56.119680 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:58.109730 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:58.109685 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:58:59.907513 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:58:59.907476 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:00.690709 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:00.690672 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:00.790733 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:00.790701 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:01.037927 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:01.037839 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k4kmq" Jun 08 22:59:01.287441 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:01.287357 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:01.388650 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:01.388571 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:02.009015 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:02.008974 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:02.103937 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:02.103898 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:02.204250 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:02.204222 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:02.251651 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:02.251626 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:04.055194 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:04.055162 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:59:05.064044 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:05.064015 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:05.166302 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:05.166264 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:09.264655 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.264616 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:09.364905 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.364878 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:09.728464 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.728386 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:09.842829 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.842792 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:09.969196 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.969162 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-625wf"] Jun 08 22:59:09.972170 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.972154 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:09.973489 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.973461 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:09.973971 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.973951 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 08 22:59:09.974071 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.974043 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ksx8k\"" Jun 08 22:59:09.974186 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.974171 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 08 22:59:09.982287 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:09.982240 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-625wf"] Jun 08 22:59:10.028661 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.028628 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9zf6\" (UniqueName: \"kubernetes.io/projected/04a9326a-0b64-40b8-8217-778d1af2668b-kube-api-access-c9zf6\") pod \"kuadrant-console-plugin-66b5774985-625wf\" (UID: \"04a9326a-0b64-40b8-8217-778d1af2668b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.028836 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.028682 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/04a9326a-0b64-40b8-8217-778d1af2668b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-625wf\" (UID: \"04a9326a-0b64-40b8-8217-778d1af2668b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.028836 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.028700 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04a9326a-0b64-40b8-8217-778d1af2668b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-625wf\" (UID: \"04a9326a-0b64-40b8-8217-778d1af2668b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.096755 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.096721 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:10.129940 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.129908 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/04a9326a-0b64-40b8-8217-778d1af2668b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-625wf\" (UID: \"04a9326a-0b64-40b8-8217-778d1af2668b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.130101 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.129948 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04a9326a-0b64-40b8-8217-778d1af2668b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-625wf\" (UID: \"04a9326a-0b64-40b8-8217-778d1af2668b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.130101 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.130007 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9zf6\" (UniqueName: \"kubernetes.io/projected/04a9326a-0b64-40b8-8217-778d1af2668b-kube-api-access-c9zf6\") pod \"kuadrant-console-plugin-66b5774985-625wf\" (UID: \"04a9326a-0b64-40b8-8217-778d1af2668b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.130624 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.130603 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04a9326a-0b64-40b8-8217-778d1af2668b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-625wf\" (UID: \"04a9326a-0b64-40b8-8217-778d1af2668b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.132525 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.132506 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/04a9326a-0b64-40b8-8217-778d1af2668b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-625wf\" (UID: \"04a9326a-0b64-40b8-8217-778d1af2668b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.137211 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.137186 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9zf6\" (UniqueName: \"kubernetes.io/projected/04a9326a-0b64-40b8-8217-778d1af2668b-kube-api-access-c9zf6\") pod \"kuadrant-console-plugin-66b5774985-625wf\" (UID: \"04a9326a-0b64-40b8-8217-778d1af2668b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.281322 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.281235 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" Jun 08 22:59:10.406549 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:10.406516 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-625wf"] Jun 08 22:59:10.410122 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:59:10.410090 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04a9326a_0b64_40b8_8217_778d1af2668b.slice/crio-d9ec729e0981dd8777d58bef4da24cc2ddd39c5374f41569687e38170f355d14 WatchSource:0}: Error finding container d9ec729e0981dd8777d58bef4da24cc2ddd39c5374f41569687e38170f355d14: Status 404 returned error can't find the container with id d9ec729e0981dd8777d58bef4da24cc2ddd39c5374f41569687e38170f355d14 Jun 08 22:59:11.138602 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:11.138561 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" event={"ID":"04a9326a-0b64-40b8-8217-778d1af2668b","Type":"ContainerStarted","Data":"d9ec729e0981dd8777d58bef4da24cc2ddd39c5374f41569687e38170f355d14"} Jun 08 22:59:13.455840 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:13.455764 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:13.555944 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:13.555910 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:20.053706 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.053604 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:20.155201 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.155167 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:20.540021 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.539981 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8"] Jun 08 22:59:20.540273 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.540245 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" podUID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" containerName="manager" containerID="cri-o://b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad" gracePeriod=2 Jun 08 22:59:20.551875 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.551847 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8"] Jun 08 22:59:20.560447 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:59:20.560370 2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8b7ec8ab-243a-4177-862f-5749bb45a8e1\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-2lxp8\": pods \"limitador-operator-controller-manager-859d7b78bd-2lxp8\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:59:20.568239 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.568202 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4"] Jun 08 22:59:20.568660 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.568643 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" containerName="manager" Jun 08 22:59:20.568749 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.568662 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" containerName="manager" Jun 08 22:59:20.568749 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.568737 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" containerName="manager" Jun 08 22:59:20.571923 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.571899 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" Jun 08 22:59:20.578064 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.578003 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4"] Jun 08 22:59:20.593841 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:59:20.593802 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2lxp8\" is forbidden: User \"system:node:ip-10-0-138-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-248.ec2.internal' and this object" podUID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:59:20.615635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.615607 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcmnb\" (UniqueName: \"kubernetes.io/projected/a1ed08a6-1795-4f77-8204-edc8808fbcbc-kube-api-access-dcmnb\") pod \"limitador-operator-controller-manager-859d7b78bd-z5cb4\" (UID: \"a1ed08a6-1795-4f77-8204-edc8808fbcbc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" Jun 08 22:59:20.716795 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.716753 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcmnb\" (UniqueName: \"kubernetes.io/projected/a1ed08a6-1795-4f77-8204-edc8808fbcbc-kube-api-access-dcmnb\") pod \"limitador-operator-controller-manager-859d7b78bd-z5cb4\" (UID: \"a1ed08a6-1795-4f77-8204-edc8808fbcbc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" Jun 08 22:59:20.725914 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.725868 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcmnb\" (UniqueName: \"kubernetes.io/projected/a1ed08a6-1795-4f77-8204-edc8808fbcbc-kube-api-access-dcmnb\") pod \"limitador-operator-controller-manager-859d7b78bd-z5cb4\" (UID: \"a1ed08a6-1795-4f77-8204-edc8808fbcbc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" Jun 08 22:59:20.911613 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:20.911533 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" Jun 08 22:59:22.880598 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:22.879953 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:22.979565 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:22.979531 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:27.664371 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:27.664324 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:27.764734 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:27.764700 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:31.192288 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:31.192247 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:31.291029 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:31.290988 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:32.708697 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:32.708672 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4"] Jun 08 22:59:32.710295 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:59:32.710266 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ed08a6_1795_4f77_8204_edc8808fbcbc.slice/crio-34425091fd9a0f0a87f39b18a7b80693b2be169afb2f6831fb662e772c6f2316 WatchSource:0}: Error finding container 34425091fd9a0f0a87f39b18a7b80693b2be169afb2f6831fb662e772c6f2316: Status 404 returned error can't find the container with id 34425091fd9a0f0a87f39b18a7b80693b2be169afb2f6831fb662e772c6f2316 Jun 08 22:59:32.718075 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:32.718053 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:59:32.719637 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:59:32.719609 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2lxp8\" is forbidden: User \"system:node:ip-10-0-138-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-248.ec2.internal' and this object" podUID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:59:32.830326 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:32.830251 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5m9m\" (UniqueName: \"kubernetes.io/projected/8b7ec8ab-243a-4177-862f-5749bb45a8e1-kube-api-access-h5m9m\") pod \"8b7ec8ab-243a-4177-862f-5749bb45a8e1\" (UID: \"8b7ec8ab-243a-4177-862f-5749bb45a8e1\") " Jun 08 22:59:32.832311 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:32.832275 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b7ec8ab-243a-4177-862f-5749bb45a8e1-kube-api-access-h5m9m" (OuterVolumeSpecName: "kube-api-access-h5m9m") pod "8b7ec8ab-243a-4177-862f-5749bb45a8e1" (UID: "8b7ec8ab-243a-4177-862f-5749bb45a8e1"). InnerVolumeSpecName "kube-api-access-h5m9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:59:32.931556 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:32.931517 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5m9m\" (UniqueName: \"kubernetes.io/projected/8b7ec8ab-243a-4177-862f-5749bb45a8e1-kube-api-access-h5m9m\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 22:59:33.273635 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.273599 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" event={"ID":"04a9326a-0b64-40b8-8217-778d1af2668b","Type":"ContainerStarted","Data":"5fb4209470771e9c668fb68f28cab79d52b6dcce9a9c8425783ffbda00039f0f"} Jun 08 22:59:33.275210 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.275180 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" event={"ID":"a1ed08a6-1795-4f77-8204-edc8808fbcbc","Type":"ContainerStarted","Data":"1769f080b362e14d8a09bf13fc48914630da7ebba2eb28b8d8c96e7d506c6c2e"} Jun 08 22:59:33.275344 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.275224 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" event={"ID":"a1ed08a6-1795-4f77-8204-edc8808fbcbc","Type":"ContainerStarted","Data":"34425091fd9a0f0a87f39b18a7b80693b2be169afb2f6831fb662e772c6f2316"} Jun 08 22:59:33.275344 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.275244 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" Jun 08 22:59:33.275534 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:59:33.275510 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2lxp8\" is forbidden: User \"system:node:ip-10-0-138-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-248.ec2.internal' and this object" podUID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:59:33.276488 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.276463 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:59:33.276589 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.276491 2567 scope.go:117] "RemoveContainer" containerID="b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad" Jun 08 22:59:33.276589 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.276431 2567 generic.go:356] "Generic (PLEG): container finished" podID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" containerID="b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad" exitCode=0 Jun 08 22:59:33.285420 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.285378 2567 scope.go:117] "RemoveContainer" containerID="b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad" Jun 08 22:59:33.285670 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:59:33.285649 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad\": container with ID starting with b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad not found: ID does not exist" containerID="b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad" Jun 08 22:59:33.285725 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.285676 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad"} err="failed to get container status \"b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad\": rpc error: code = NotFound desc = could not find container \"b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad\": container with ID starting with b22194bbe46dd5106d825f5e6671741a14e0e9e715cb8cf64090a458913b91ad not found: ID does not exist" Jun 08 22:59:33.297531 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.297493 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-625wf" podStartSLOduration=2.047873185 podStartE2EDuration="24.297480587s" totalImagesPullingTime="22.249607402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:59:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:59:33.295915257 +0000 UTC m=+695.104527135" watchObservedRunningTime="2026-06-08 22:59:33.297480587 +0000 UTC m=+695.106092495" Jun 08 22:59:33.315358 ip-10-0-138-248 kubenswrapper[2567]: E0608 22:59:33.315322 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2lxp8\" is forbidden: User \"system:node:ip-10-0-138-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-248.ec2.internal' and this object" podUID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2lxp8" Jun 08 22:59:33.315733 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:33.315690 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" podStartSLOduration=13.31567714 podStartE2EDuration="13.31567714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:59:33.313359861 +0000 UTC m=+695.121971715" watchObservedRunningTime="2026-06-08 22:59:33.31567714 +0000 UTC m=+695.124288993" Jun 08 22:59:34.621078 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:34.621043 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b7ec8ab-243a-4177-862f-5749bb45a8e1" path="/var/lib/kubelet/pods/8b7ec8ab-243a-4177-862f-5749bb45a8e1/volumes" Jun 08 22:59:35.383781 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:35.383751 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:35.484555 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:35.484524 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:37.836035 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:37.836010 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:37.937749 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:37.937724 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:41.036681 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:41.036642 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:41.141507 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:41.141475 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:44.288015 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:44.287990 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z5cb4" Jun 08 22:59:45.465448 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:45.465413 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:45.566128 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:45.566094 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:46.434681 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:46.434650 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:46.487036 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:46.486996 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:49.223531 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.223502 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq"] Jun 08 22:59:49.234033 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.234006 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.236595 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.236571 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-cfst7\"" Jun 08 22:59:49.236840 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.236819 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq"] Jun 08 22:59:49.258836 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.258809 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:49.317371 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.317342 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:49.379051 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.379023 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.379051 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.379053 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.379247 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.379082 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.379247 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.379150 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.379247 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.379180 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcbwd\" (UniqueName: \"kubernetes.io/projected/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-kube-api-access-bcbwd\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.379247 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.379202 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.379247 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.379242 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.379466 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.379266 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.379466 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.379298 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.418563 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.418530 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:49.480751 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.480676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcbwd\" (UniqueName: \"kubernetes.io/projected/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-kube-api-access-bcbwd\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.480751 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.480719 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.480948 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.480761 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.480948 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.480785 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.480948 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.480830 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.481096 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.480982 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.481096 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.481041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.481196 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.481101 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.481196 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.481148 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.481296 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.481201 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.481343 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.481309 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.481538 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.481509 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.481538 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.481528 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.481727 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.481668 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.483105 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.483086 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.483515 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.483495 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.488686 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.488659 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.488768 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.488718 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcbwd\" (UniqueName: \"kubernetes.io/projected/ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6-kube-api-access-bcbwd\") pod \"maas-default-gateway-openshift-default-8559cd5744-zggbq\" (UID: \"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.518720 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.518695 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 22:59:49.547867 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.547832 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:49.678834 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.678802 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq"] Jun 08 22:59:49.680129 ip-10-0-138-248 kubenswrapper[2567]: W0608 22:59:49.680097 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab03d9ec_73be_48e6_9fa8_ae7fd9c97ad6.slice/crio-fe5c938952517339c43ef080d0d5a1a2418bec53e93de8107ccc4b9739d353f5 WatchSource:0}: Error finding container fe5c938952517339c43ef080d0d5a1a2418bec53e93de8107ccc4b9739d353f5: Status 404 returned error can't find the container with id fe5c938952517339c43ef080d0d5a1a2418bec53e93de8107ccc4b9739d353f5 Jun 08 22:59:49.682337 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.682302 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 22:59:49.682428 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.682386 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 22:59:49.682474 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:49.682456 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 22:59:50.363839 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:50.363800 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" event={"ID":"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6","Type":"ContainerStarted","Data":"b160d3a7c4ac25ff0b5ba418be363f2ba273dc1e39d5fe1b897906172b976b0c"} Jun 08 22:59:50.364255 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:50.363890 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" event={"ID":"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6","Type":"ContainerStarted","Data":"fe5c938952517339c43ef080d0d5a1a2418bec53e93de8107ccc4b9739d353f5"} Jun 08 22:59:50.383168 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:50.383106 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" podStartSLOduration=1.3830931 podStartE2EDuration="1.3830931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:59:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:59:50.38018216 +0000 UTC m=+712.188794014" watchObservedRunningTime="2026-06-08 22:59:50.3830931 +0000 UTC m=+712.191704953" Jun 08 22:59:50.548654 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:50.548620 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:50.553673 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:50.553646 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:51.369678 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:51.369646 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:51.370767 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:51.370748 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 22:59:52.797203 ip-10-0-138-248 kubenswrapper[2567]: I0608 22:59:52.797156 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:03.786916 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.786881 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:03.790430 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.790376 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-25gks"] Jun 08 23:00:03.821235 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.821196 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-25gks"] Jun 08 23:00:03.821409 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.821313 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:03.823578 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.823556 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 08 23:00:03.884615 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.884580 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-25gks"] Jun 08 23:00:03.886705 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.886680 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:03.900853 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.900820 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec0a8de9-c67c-4063-b726-f51475eb6b4d-config-file\") pod \"limitador-limitador-64c94fd64f-25gks\" (UID: \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:03.901004 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.900892 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8jms\" (UniqueName: \"kubernetes.io/projected/ec0a8de9-c67c-4063-b726-f51475eb6b4d-kube-api-access-v8jms\") pod \"limitador-limitador-64c94fd64f-25gks\" (UID: \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:03.987616 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:03.987585 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:04.001743 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.001715 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8jms\" (UniqueName: \"kubernetes.io/projected/ec0a8de9-c67c-4063-b726-f51475eb6b4d-kube-api-access-v8jms\") pod \"limitador-limitador-64c94fd64f-25gks\" (UID: \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:04.001854 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.001761 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec0a8de9-c67c-4063-b726-f51475eb6b4d-config-file\") pod \"limitador-limitador-64c94fd64f-25gks\" (UID: \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:04.002307 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.002291 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec0a8de9-c67c-4063-b726-f51475eb6b4d-config-file\") pod \"limitador-limitador-64c94fd64f-25gks\" (UID: \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:04.008531 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.008510 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8jms\" (UniqueName: \"kubernetes.io/projected/ec0a8de9-c67c-4063-b726-f51475eb6b4d-kube-api-access-v8jms\") pod \"limitador-limitador-64c94fd64f-25gks\" (UID: \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:04.131329 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.131245 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:04.262437 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.262410 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-25gks"] Jun 08 23:00:04.264434 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:00:04.264376 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0a8de9_c67c_4063_b726_f51475eb6b4d.slice/crio-79089122d4ac9c2c1924d2a09d47b886d5f86669383380ea48a5db4ac8f0a431 WatchSource:0}: Error finding container 79089122d4ac9c2c1924d2a09d47b886d5f86669383380ea48a5db4ac8f0a431: Status 404 returned error can't find the container with id 79089122d4ac9c2c1924d2a09d47b886d5f86669383380ea48a5db4ac8f0a431 Jun 08 23:00:04.435155 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.435111 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" event={"ID":"ec0a8de9-c67c-4063-b726-f51475eb6b4d","Type":"ContainerStarted","Data":"79089122d4ac9c2c1924d2a09d47b886d5f86669383380ea48a5db4ac8f0a431"} Jun 08 23:00:04.550574 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.550542 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-sgrmh"] Jun 08 23:00:04.554859 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.554838 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-sgrmh" Jun 08 23:00:04.556821 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.556792 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l5nsf\"" Jun 08 23:00:04.559175 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.559153 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-sgrmh"] Jun 08 23:00:04.582727 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.582695 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:04.606711 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.606675 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c82pp\" (UniqueName: \"kubernetes.io/projected/7a1a2646-544e-4a5b-bfe4-762304495777-kube-api-access-c82pp\") pod \"authorino-58cd955675-sgrmh\" (UID: \"7a1a2646-544e-4a5b-bfe4-762304495777\") " pod="kuadrant-system/authorino-58cd955675-sgrmh" Jun 08 23:00:04.635276 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.635248 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:04.706330 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.706240 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hnmhj"] Jun 08 23:00:04.707427 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.707384 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c82pp\" (UniqueName: \"kubernetes.io/projected/7a1a2646-544e-4a5b-bfe4-762304495777-kube-api-access-c82pp\") pod \"authorino-58cd955675-sgrmh\" (UID: \"7a1a2646-544e-4a5b-bfe4-762304495777\") " pod="kuadrant-system/authorino-58cd955675-sgrmh" Jun 08 23:00:04.709671 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.709650 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hnmhj" Jun 08 23:00:04.715994 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.715941 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hnmhj"] Jun 08 23:00:04.719387 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.719326 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c82pp\" (UniqueName: \"kubernetes.io/projected/7a1a2646-544e-4a5b-bfe4-762304495777-kube-api-access-c82pp\") pod \"authorino-58cd955675-sgrmh\" (UID: \"7a1a2646-544e-4a5b-bfe4-762304495777\") " pod="kuadrant-system/authorino-58cd955675-sgrmh" Jun 08 23:00:04.808461 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.808420 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vd22\" (UniqueName: \"kubernetes.io/projected/9afe1e81-cf0a-4dea-b466-43d768d9af77-kube-api-access-2vd22\") pod \"authorino-75d556bd77-hnmhj\" (UID: \"9afe1e81-cf0a-4dea-b466-43d768d9af77\") " pod="kuadrant-system/authorino-75d556bd77-hnmhj" Jun 08 23:00:04.865328 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.865288 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-sgrmh" Jun 08 23:00:04.909940 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.909898 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vd22\" (UniqueName: \"kubernetes.io/projected/9afe1e81-cf0a-4dea-b466-43d768d9af77-kube-api-access-2vd22\") pod \"authorino-75d556bd77-hnmhj\" (UID: \"9afe1e81-cf0a-4dea-b466-43d768d9af77\") " pod="kuadrant-system/authorino-75d556bd77-hnmhj" Jun 08 23:00:04.918012 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:04.917982 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vd22\" (UniqueName: \"kubernetes.io/projected/9afe1e81-cf0a-4dea-b466-43d768d9af77-kube-api-access-2vd22\") pod \"authorino-75d556bd77-hnmhj\" (UID: \"9afe1e81-cf0a-4dea-b466-43d768d9af77\") " pod="kuadrant-system/authorino-75d556bd77-hnmhj" Jun 08 23:00:05.020927 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:05.020891 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hnmhj" Jun 08 23:00:05.022415 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:05.022318 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-sgrmh"] Jun 08 23:00:05.029191 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:00:05.029147 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a1a2646_544e_4a5b_bfe4_762304495777.slice/crio-538c86c59376b6575834035d8be7063b3f0b57259bec8f5d3f2f759eb05ca4b0 WatchSource:0}: Error finding container 538c86c59376b6575834035d8be7063b3f0b57259bec8f5d3f2f759eb05ca4b0: Status 404 returned error can't find the container with id 538c86c59376b6575834035d8be7063b3f0b57259bec8f5d3f2f759eb05ca4b0 Jun 08 23:00:05.207021 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:05.206983 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hnmhj"] Jun 08 23:00:05.443100 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:05.443057 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hnmhj" event={"ID":"9afe1e81-cf0a-4dea-b466-43d768d9af77","Type":"ContainerStarted","Data":"c218946918d3a32c7802dd4c4b5ff5f71afe450d302941f9c8114b863e8c91de"} Jun 08 23:00:05.444365 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:05.444333 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-sgrmh" event={"ID":"7a1a2646-544e-4a5b-bfe4-762304495777","Type":"ContainerStarted","Data":"538c86c59376b6575834035d8be7063b3f0b57259bec8f5d3f2f759eb05ca4b0"} Jun 08 23:00:10.480150 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:10.480110 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hnmhj" event={"ID":"9afe1e81-cf0a-4dea-b466-43d768d9af77","Type":"ContainerStarted","Data":"04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb"} Jun 08 23:00:10.481653 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:10.481625 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" event={"ID":"ec0a8de9-c67c-4063-b726-f51475eb6b4d","Type":"ContainerStarted","Data":"da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc"} Jun 08 23:00:10.481801 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:10.481696 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:10.483034 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:10.483002 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-sgrmh" event={"ID":"7a1a2646-544e-4a5b-bfe4-762304495777","Type":"ContainerStarted","Data":"0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11"} Jun 08 23:00:10.492960 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:10.492906 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hnmhj" podStartSLOduration=2.176114675 podStartE2EDuration="6.492891907s" totalImagesPullingTime="4.316777232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:00:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:00:10.491756857 +0000 UTC m=+732.300368710" watchObservedRunningTime="2026-06-08 23:00:10.492891907 +0000 UTC m=+732.301503760" Jun 08 23:00:10.508503 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:10.508447 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" podStartSLOduration=2.30455497 podStartE2EDuration="7.508427581s" totalImagesPullingTime="5.203872611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:00:10.505238158 +0000 UTC m=+732.313850013" watchObservedRunningTime="2026-06-08 23:00:10.508427581 +0000 UTC m=+732.317039435" Jun 08 23:00:10.519557 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:10.519529 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-sgrmh"] Jun 08 23:00:10.529082 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:10.529028 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-sgrmh" podStartSLOduration=2.018404514 podStartE2EDuration="6.529007414s" totalImagesPullingTime="4.5106029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:00:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:00:10.526825394 +0000 UTC m=+732.335437251" watchObservedRunningTime="2026-06-08 23:00:10.529007414 +0000 UTC m=+732.337619273" Jun 08 23:00:12.498752 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:12.498689 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-sgrmh" podUID="7a1a2646-544e-4a5b-bfe4-762304495777" containerName="authorino" containerID="cri-o://0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11" gracePeriod=30 Jun 08 23:00:12.743543 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:12.743519 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-sgrmh" Jun 08 23:00:12.788337 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:12.788263 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c82pp\" (UniqueName: \"kubernetes.io/projected/7a1a2646-544e-4a5b-bfe4-762304495777-kube-api-access-c82pp\") pod \"7a1a2646-544e-4a5b-bfe4-762304495777\" (UID: \"7a1a2646-544e-4a5b-bfe4-762304495777\") " Jun 08 23:00:12.790231 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:12.790204 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a1a2646-544e-4a5b-bfe4-762304495777-kube-api-access-c82pp" (OuterVolumeSpecName: "kube-api-access-c82pp") pod "7a1a2646-544e-4a5b-bfe4-762304495777" (UID: "7a1a2646-544e-4a5b-bfe4-762304495777"). InnerVolumeSpecName "kube-api-access-c82pp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:00:12.889512 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:12.889482 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c82pp\" (UniqueName: \"kubernetes.io/projected/7a1a2646-544e-4a5b-bfe4-762304495777-kube-api-access-c82pp\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:00:13.505579 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:13.505540 2567 generic.go:356] "Generic (PLEG): container finished" podID="7a1a2646-544e-4a5b-bfe4-762304495777" containerID="0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11" exitCode=0 Jun 08 23:00:13.505579 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:13.505584 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-sgrmh" Jun 08 23:00:13.506024 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:13.505594 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-sgrmh" event={"ID":"7a1a2646-544e-4a5b-bfe4-762304495777","Type":"ContainerDied","Data":"0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11"} Jun 08 23:00:13.506024 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:13.505633 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-sgrmh" event={"ID":"7a1a2646-544e-4a5b-bfe4-762304495777","Type":"ContainerDied","Data":"538c86c59376b6575834035d8be7063b3f0b57259bec8f5d3f2f759eb05ca4b0"} Jun 08 23:00:13.506024 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:13.505655 2567 scope.go:117] "RemoveContainer" containerID="0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11" Jun 08 23:00:13.514734 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:13.514712 2567 scope.go:117] "RemoveContainer" containerID="0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11" Jun 08 23:00:13.514985 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:00:13.514967 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11\": container with ID starting with 0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11 not found: ID does not exist" containerID="0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11" Jun 08 23:00:13.515063 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:13.514993 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11"} err="failed to get container status \"0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11\": rpc error: code = NotFound desc = could not find container \"0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11\": container with ID starting with 0d6994ca5cdfaef206a126d659519f3685df49ce4d317d204120d1f0f567bd11 not found: ID does not exist" Jun 08 23:00:13.524457 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:13.524431 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-sgrmh"] Jun 08 23:00:13.527595 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:13.527573 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-sgrmh"] Jun 08 23:00:14.621211 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:14.621179 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a1a2646-544e-4a5b-bfe4-762304495777" path="/var/lib/kubelet/pods/7a1a2646-544e-4a5b-bfe4-762304495777/volumes" Jun 08 23:00:17.459253 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:17.459226 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:17.558883 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:17.558847 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:18.262599 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:18.262560 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-25gks"] Jun 08 23:00:18.262808 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:18.262761 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" podUID="ec0a8de9-c67c-4063-b726-f51475eb6b4d" containerName="limitador" containerID="cri-o://da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc" gracePeriod=30 Jun 08 23:00:18.263473 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:18.263440 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:18.831425 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:18.831349 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:18.940861 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:18.940828 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec0a8de9-c67c-4063-b726-f51475eb6b4d-config-file\") pod \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\" (UID: \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\") " Jun 08 23:00:18.941000 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:18.940901 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8jms\" (UniqueName: \"kubernetes.io/projected/ec0a8de9-c67c-4063-b726-f51475eb6b4d-kube-api-access-v8jms\") pod \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\" (UID: \"ec0a8de9-c67c-4063-b726-f51475eb6b4d\") " Jun 08 23:00:18.941206 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:18.941182 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec0a8de9-c67c-4063-b726-f51475eb6b4d-config-file" (OuterVolumeSpecName: "config-file") pod "ec0a8de9-c67c-4063-b726-f51475eb6b4d" (UID: "ec0a8de9-c67c-4063-b726-f51475eb6b4d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 23:00:18.942893 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:18.942871 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec0a8de9-c67c-4063-b726-f51475eb6b4d-kube-api-access-v8jms" (OuterVolumeSpecName: "kube-api-access-v8jms") pod "ec0a8de9-c67c-4063-b726-f51475eb6b4d" (UID: "ec0a8de9-c67c-4063-b726-f51475eb6b4d"). InnerVolumeSpecName "kube-api-access-v8jms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:00:19.041833 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.041792 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8jms\" (UniqueName: \"kubernetes.io/projected/ec0a8de9-c67c-4063-b726-f51475eb6b4d-kube-api-access-v8jms\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:00:19.041833 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.041825 2567 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec0a8de9-c67c-4063-b726-f51475eb6b4d-config-file\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:00:19.460852 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.460820 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:19.537753 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.537715 2567 generic.go:356] "Generic (PLEG): container finished" podID="ec0a8de9-c67c-4063-b726-f51475eb6b4d" containerID="da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc" exitCode=0 Jun 08 23:00:19.537917 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.537761 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" Jun 08 23:00:19.537917 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.537796 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" event={"ID":"ec0a8de9-c67c-4063-b726-f51475eb6b4d","Type":"ContainerDied","Data":"da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc"} Jun 08 23:00:19.537917 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.537849 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-25gks" event={"ID":"ec0a8de9-c67c-4063-b726-f51475eb6b4d","Type":"ContainerDied","Data":"79089122d4ac9c2c1924d2a09d47b886d5f86669383380ea48a5db4ac8f0a431"} Jun 08 23:00:19.537917 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.537872 2567 scope.go:117] "RemoveContainer" containerID="da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc" Jun 08 23:00:19.546411 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.546372 2567 scope.go:117] "RemoveContainer" containerID="da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc" Jun 08 23:00:19.546661 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:00:19.546642 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc\": container with ID starting with da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc not found: ID does not exist" containerID="da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc" Jun 08 23:00:19.546707 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.546670 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc"} err="failed to get container status \"da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc\": rpc error: code = NotFound desc = could not find container \"da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc\": container with ID starting with da7aafe7ba89132db6cdee724689a904f85e3763f4e2ccf7745401dba20181cc not found: ID does not exist" Jun 08 23:00:19.557832 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.557808 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-25gks"] Jun 08 23:00:19.561206 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.561134 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:19.563700 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.563681 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-25gks"] Jun 08 23:00:19.889837 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.889752 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f"] Jun 08 23:00:19.890284 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.890198 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a1a2646-544e-4a5b-bfe4-762304495777" containerName="authorino" Jun 08 23:00:19.890284 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.890218 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a1a2646-544e-4a5b-bfe4-762304495777" containerName="authorino" Jun 08 23:00:19.890284 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.890249 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec0a8de9-c67c-4063-b726-f51475eb6b4d" containerName="limitador" Jun 08 23:00:19.890284 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.890258 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec0a8de9-c67c-4063-b726-f51475eb6b4d" containerName="limitador" Jun 08 23:00:19.890587 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.890342 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec0a8de9-c67c-4063-b726-f51475eb6b4d" containerName="limitador" Jun 08 23:00:19.890587 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.890359 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a1a2646-544e-4a5b-bfe4-762304495777" containerName="authorino" Jun 08 23:00:19.894356 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.894336 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:19.896312 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.896291 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jun 08 23:00:19.896459 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.896322 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"postgres-creds\"" Jun 08 23:00:19.896459 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.896325 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jun 08 23:00:19.896590 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.896454 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"default-dockercfg-qrtpx\"" Jun 08 23:00:19.900325 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.900302 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f"] Jun 08 23:00:19.932722 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.932699 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:19.949319 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.949294 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25lvd\" (UniqueName: \"kubernetes.io/projected/a6047700-a2b1-420d-a50c-db4fd636c876-kube-api-access-25lvd\") pod \"postgres-69fb4594cf-8jj9f\" (UID: \"a6047700-a2b1-420d-a50c-db4fd636c876\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:19.949466 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:19.949362 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a6047700-a2b1-420d-a50c-db4fd636c876-data\") pod \"postgres-69fb4594cf-8jj9f\" (UID: \"a6047700-a2b1-420d-a50c-db4fd636c876\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:20.050046 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:20.050007 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a6047700-a2b1-420d-a50c-db4fd636c876-data\") pod \"postgres-69fb4594cf-8jj9f\" (UID: \"a6047700-a2b1-420d-a50c-db4fd636c876\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:20.050237 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:20.050072 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25lvd\" (UniqueName: \"kubernetes.io/projected/a6047700-a2b1-420d-a50c-db4fd636c876-kube-api-access-25lvd\") pod \"postgres-69fb4594cf-8jj9f\" (UID: \"a6047700-a2b1-420d-a50c-db4fd636c876\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:20.050438 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:20.050418 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a6047700-a2b1-420d-a50c-db4fd636c876-data\") pod \"postgres-69fb4594cf-8jj9f\" (UID: \"a6047700-a2b1-420d-a50c-db4fd636c876\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:20.057478 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:20.057460 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25lvd\" (UniqueName: \"kubernetes.io/projected/a6047700-a2b1-420d-a50c-db4fd636c876-kube-api-access-25lvd\") pod \"postgres-69fb4594cf-8jj9f\" (UID: \"a6047700-a2b1-420d-a50c-db4fd636c876\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:20.206662 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:20.206625 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:20.335320 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:20.335291 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f"] Jun 08 23:00:20.336912 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:00:20.336884 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6047700_a2b1_420d_a50c_db4fd636c876.slice/crio-83cd27709be01a4aeb92d6ee47a940515c3f684feac6fb2972950a8b1a51131e WatchSource:0}: Error finding container 83cd27709be01a4aeb92d6ee47a940515c3f684feac6fb2972950a8b1a51131e: Status 404 returned error can't find the container with id 83cd27709be01a4aeb92d6ee47a940515c3f684feac6fb2972950a8b1a51131e Jun 08 23:00:20.543375 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:20.543283 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" event={"ID":"a6047700-a2b1-420d-a50c-db4fd636c876","Type":"ContainerStarted","Data":"83cd27709be01a4aeb92d6ee47a940515c3f684feac6fb2972950a8b1a51131e"} Jun 08 23:00:20.621161 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:20.621128 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec0a8de9-c67c-4063-b726-f51475eb6b4d" path="/var/lib/kubelet/pods/ec0a8de9-c67c-4063-b726-f51475eb6b4d/volumes" Jun 08 23:00:24.565959 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:24.565923 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" event={"ID":"a6047700-a2b1-420d-a50c-db4fd636c876","Type":"ContainerStarted","Data":"ad519af92ccfe8108ed0c1f379ac56c856c54d2d28081795b593e17f0eeeb784"} Jun 08 23:00:24.566351 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:24.566054 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:24.581792 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:24.581744 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" podStartSLOduration=1.8295671850000002 podStartE2EDuration="5.581731377s" totalImagesPullingTime="3.752164192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:00:24.579005401 +0000 UTC m=+746.387617255" watchObservedRunningTime="2026-06-08 23:00:24.581731377 +0000 UTC m=+746.390343231" Jun 08 23:00:30.603528 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:30.603498 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-8jj9f" Jun 08 23:00:30.934191 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:30.934161 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:31.033160 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:31.033127 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:31.150037 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:31.150005 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:31.907070 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:31.907035 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d"] Jun 08 23:00:31.911903 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:31.911886 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:31.914003 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:31.913976 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 08 23:00:31.914118 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:31.914022 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 08 23:00:31.914573 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:31.914552 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7qzv5\"" Jun 08 23:00:31.918439 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:31.918416 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d"] Jun 08 23:00:32.052195 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.052152 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.052195 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.052194 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blhvn\" (UniqueName: \"kubernetes.io/projected/5ba11825-22fd-4ded-bdcc-179ec7049a12-kube-api-access-blhvn\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.052458 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.052224 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.153632 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.153592 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.153632 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.153634 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blhvn\" (UniqueName: \"kubernetes.io/projected/5ba11825-22fd-4ded-bdcc-179ec7049a12-kube-api-access-blhvn\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.153887 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.153679 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.153984 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.153963 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.154079 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.154060 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.161426 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.161349 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blhvn\" (UniqueName: \"kubernetes.io/projected/5ba11825-22fd-4ded-bdcc-179ec7049a12-kube-api-access-blhvn\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.223065 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.223030 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:32.352235 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.352210 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d"] Jun 08 23:00:32.354151 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:00:32.354123 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba11825_22fd_4ded_bdcc_179ec7049a12.slice/crio-6d55c49b6797456c160bc863992ad12968d439a2dea8fa21f7f87cb650120bc9 WatchSource:0}: Error finding container 6d55c49b6797456c160bc863992ad12968d439a2dea8fa21f7f87cb650120bc9: Status 404 returned error can't find the container with id 6d55c49b6797456c160bc863992ad12968d439a2dea8fa21f7f87cb650120bc9 Jun 08 23:00:32.606888 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.606857 2567 generic.go:356] "Generic (PLEG): container finished" podID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerID="bb61df52a0aa49e71f618bd7094ead4543253957dffece0119a29bf109b89646" exitCode=0 Jun 08 23:00:32.607080 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.606894 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" event={"ID":"5ba11825-22fd-4ded-bdcc-179ec7049a12","Type":"ContainerDied","Data":"bb61df52a0aa49e71f618bd7094ead4543253957dffece0119a29bf109b89646"} Jun 08 23:00:32.607080 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:32.606948 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" event={"ID":"5ba11825-22fd-4ded-bdcc-179ec7049a12","Type":"ContainerStarted","Data":"6d55c49b6797456c160bc863992ad12968d439a2dea8fa21f7f87cb650120bc9"} Jun 08 23:00:33.613630 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:33.613536 2567 generic.go:356] "Generic (PLEG): container finished" podID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerID="664cd23c40e98b24f6f454c5fbf0d4190f76236b10ce910a5e08cfeae77543a6" exitCode=0 Jun 08 23:00:33.613630 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:33.613618 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" event={"ID":"5ba11825-22fd-4ded-bdcc-179ec7049a12","Type":"ContainerDied","Data":"664cd23c40e98b24f6f454c5fbf0d4190f76236b10ce910a5e08cfeae77543a6"} Jun 08 23:00:34.621870 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:34.621839 2567 generic.go:356] "Generic (PLEG): container finished" podID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerID="d4a594e63907a6f9e93f9f0117932531560d922423aaefed19ae01b112e9da72" exitCode=0 Jun 08 23:00:34.622252 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:34.622096 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" event={"ID":"5ba11825-22fd-4ded-bdcc-179ec7049a12","Type":"ContainerDied","Data":"d4a594e63907a6f9e93f9f0117932531560d922423aaefed19ae01b112e9da72"} Jun 08 23:00:35.773121 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.773089 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:35.888607 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.888532 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-util\") pod \"5ba11825-22fd-4ded-bdcc-179ec7049a12\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " Jun 08 23:00:35.888607 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.888565 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-bundle\") pod \"5ba11825-22fd-4ded-bdcc-179ec7049a12\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " Jun 08 23:00:35.888607 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.888599 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blhvn\" (UniqueName: \"kubernetes.io/projected/5ba11825-22fd-4ded-bdcc-179ec7049a12-kube-api-access-blhvn\") pod \"5ba11825-22fd-4ded-bdcc-179ec7049a12\" (UID: \"5ba11825-22fd-4ded-bdcc-179ec7049a12\") " Jun 08 23:00:35.889012 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.888980 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-bundle" (OuterVolumeSpecName: "bundle") pod "5ba11825-22fd-4ded-bdcc-179ec7049a12" (UID: "5ba11825-22fd-4ded-bdcc-179ec7049a12"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 23:00:35.890719 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.890695 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba11825-22fd-4ded-bdcc-179ec7049a12-kube-api-access-blhvn" (OuterVolumeSpecName: "kube-api-access-blhvn") pod "5ba11825-22fd-4ded-bdcc-179ec7049a12" (UID: "5ba11825-22fd-4ded-bdcc-179ec7049a12"). InnerVolumeSpecName "kube-api-access-blhvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:00:35.894132 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.894111 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-util" (OuterVolumeSpecName: "util") pod "5ba11825-22fd-4ded-bdcc-179ec7049a12" (UID: "5ba11825-22fd-4ded-bdcc-179ec7049a12"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 23:00:35.990046 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.990003 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-util\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:00:35.990046 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.990046 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ba11825-22fd-4ded-bdcc-179ec7049a12-bundle\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:00:35.990253 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:35.990062 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blhvn\" (UniqueName: \"kubernetes.io/projected/5ba11825-22fd-4ded-bdcc-179ec7049a12-kube-api-access-blhvn\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:00:36.633544 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:36.633519 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" Jun 08 23:00:36.633679 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:36.633541 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da74889h8d" event={"ID":"5ba11825-22fd-4ded-bdcc-179ec7049a12","Type":"ContainerDied","Data":"6d55c49b6797456c160bc863992ad12968d439a2dea8fa21f7f87cb650120bc9"} Jun 08 23:00:36.633679 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:36.633589 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d55c49b6797456c160bc863992ad12968d439a2dea8fa21f7f87cb650120bc9" Jun 08 23:00:36.947589 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:36.947559 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:37.005607 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:37.005573 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:55.110151 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.110121 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:55.165336 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.165303 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:00:55.515937 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.515904 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 23:00:55.516223 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.516211 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerName="util" Jun 08 23:00:55.516261 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.516225 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerName="util" Jun 08 23:00:55.516261 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.516232 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerName="extract" Jun 08 23:00:55.516261 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.516238 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerName="extract" Jun 08 23:00:55.516261 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.516249 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerName="pull" Jun 08 23:00:55.516261 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.516254 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerName="pull" Jun 08 23:00:55.516459 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.516318 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ba11825-22fd-4ded-bdcc-179ec7049a12" containerName="extract" Jun 08 23:00:55.519254 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.519237 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 23:00:55.521142 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.521114 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 08 23:00:55.521292 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.521121 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 08 23:00:55.521292 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.521256 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-lpxjq\"" Jun 08 23:00:55.521728 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.521710 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 08 23:00:55.528512 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.528481 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 23:00:55.549523 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.549497 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdwvq\" (UniqueName: \"kubernetes.io/projected/9726d38f-8d3e-4a49-8461-28a0c0ecb0bf-kube-api-access-vdwvq\") pod \"maas-keycloak-0\" (UID: \"9726d38f-8d3e-4a49-8461-28a0c0ecb0bf\") " pod="keycloak-system/maas-keycloak-0" Jun 08 23:00:55.650467 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.650429 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdwvq\" (UniqueName: \"kubernetes.io/projected/9726d38f-8d3e-4a49-8461-28a0c0ecb0bf-kube-api-access-vdwvq\") pod \"maas-keycloak-0\" (UID: \"9726d38f-8d3e-4a49-8461-28a0c0ecb0bf\") " pod="keycloak-system/maas-keycloak-0" Jun 08 23:00:55.658019 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.657984 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdwvq\" (UniqueName: \"kubernetes.io/projected/9726d38f-8d3e-4a49-8461-28a0c0ecb0bf-kube-api-access-vdwvq\") pod \"maas-keycloak-0\" (UID: \"9726d38f-8d3e-4a49-8461-28a0c0ecb0bf\") " pod="keycloak-system/maas-keycloak-0" Jun 08 23:00:55.830115 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.830014 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 23:00:55.959159 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:55.959129 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 23:00:55.960181 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:00:55.960156 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9726d38f_8d3e_4a49_8461_28a0c0ecb0bf.slice/crio-dd68ba89eb9697523958da8e4767e10c4608ece143715744270afda6ed617bb6 WatchSource:0}: Error finding container dd68ba89eb9697523958da8e4767e10c4608ece143715744270afda6ed617bb6: Status 404 returned error can't find the container with id dd68ba89eb9697523958da8e4767e10c4608ece143715744270afda6ed617bb6 Jun 08 23:00:56.742468 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:00:56.742431 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9726d38f-8d3e-4a49-8461-28a0c0ecb0bf","Type":"ContainerStarted","Data":"dd68ba89eb9697523958da8e4767e10c4608ece143715744270afda6ed617bb6"} Jun 08 23:01:00.312294 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:00.312273 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 08 23:01:00.770634 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:00.770590 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9726d38f-8d3e-4a49-8461-28a0c0ecb0bf","Type":"ContainerStarted","Data":"7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733"} Jun 08 23:01:00.787455 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:00.787376 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.438927962 podStartE2EDuration="5.787354626s" totalImagesPullingTime="4.348426664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:01:00.785439325 +0000 UTC m=+782.594051180" watchObservedRunningTime="2026-06-08 23:01:00.787354626 +0000 UTC m=+782.595966481" Jun 08 23:01:00.830815 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:00.830760 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 08 23:01:00.832735 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:00.832690 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:01.831522 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:01.831466 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:02.831096 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:02.831038 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:03.831093 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:03.831035 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:04.831143 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:04.831092 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:05.830904 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:05.830863 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 08 23:01:05.831202 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:05.831169 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:06.831197 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:06.831146 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:07.830810 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:07.830759 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:08.831061 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:08.831021 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:09.830591 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:09.830536 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:10.830827 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:10.830772 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:11.830674 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:11.830616 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:12.830596 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:12.830552 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused" Jun 08 23:01:13.957621 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:13.957570 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 08 23:01:13.972440 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:13.972355 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 23:01:23.963916 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:23.963881 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 08 23:01:24.595962 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:24.595933 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:01:24.827152 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:24.827119 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hgff9"] Jun 08 23:01:24.909557 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:24.909482 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hgff9"] Jun 08 23:01:24.909696 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:24.909582 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hgff9" Jun 08 23:01:25.048197 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.048164 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wkg8\" (UniqueName: \"kubernetes.io/projected/e63b93db-9fb1-42f5-95a6-2e1f2e62fa55-kube-api-access-9wkg8\") pod \"authorino-5d4b67c446-hgff9\" (UID: \"e63b93db-9fb1-42f5-95a6-2e1f2e62fa55\") " pod="kuadrant-system/authorino-5d4b67c446-hgff9" Jun 08 23:01:25.054622 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.054597 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hgff9"] Jun 08 23:01:25.054833 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:01:25.054815 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9wkg8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-hgff9" podUID="e63b93db-9fb1-42f5-95a6-2e1f2e62fa55" Jun 08 23:01:25.078473 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.078448 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65685c7f4d-94cnb"] Jun 08 23:01:25.102173 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.102150 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65685c7f4d-94cnb"] Jun 08 23:01:25.102281 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.102233 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65685c7f4d-94cnb" Jun 08 23:01:25.148627 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.148595 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wkg8\" (UniqueName: \"kubernetes.io/projected/e63b93db-9fb1-42f5-95a6-2e1f2e62fa55-kube-api-access-9wkg8\") pod \"authorino-5d4b67c446-hgff9\" (UID: \"e63b93db-9fb1-42f5-95a6-2e1f2e62fa55\") " pod="kuadrant-system/authorino-5d4b67c446-hgff9" Jun 08 23:01:25.155340 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.155312 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wkg8\" (UniqueName: \"kubernetes.io/projected/e63b93db-9fb1-42f5-95a6-2e1f2e62fa55-kube-api-access-9wkg8\") pod \"authorino-5d4b67c446-hgff9\" (UID: \"e63b93db-9fb1-42f5-95a6-2e1f2e62fa55\") " pod="kuadrant-system/authorino-5d4b67c446-hgff9" Jun 08 23:01:25.249502 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.249460 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhlls\" (UniqueName: \"kubernetes.io/projected/be5e8a2f-2eea-4768-96ad-84dfa11272be-kube-api-access-qhlls\") pod \"authorino-65685c7f4d-94cnb\" (UID: \"be5e8a2f-2eea-4768-96ad-84dfa11272be\") " pod="kuadrant-system/authorino-65685c7f4d-94cnb" Jun 08 23:01:25.350681 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.350649 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhlls\" (UniqueName: \"kubernetes.io/projected/be5e8a2f-2eea-4768-96ad-84dfa11272be-kube-api-access-qhlls\") pod \"authorino-65685c7f4d-94cnb\" (UID: \"be5e8a2f-2eea-4768-96ad-84dfa11272be\") " pod="kuadrant-system/authorino-65685c7f4d-94cnb" Jun 08 23:01:25.357806 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.357776 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhlls\" (UniqueName: \"kubernetes.io/projected/be5e8a2f-2eea-4768-96ad-84dfa11272be-kube-api-access-qhlls\") pod \"authorino-65685c7f4d-94cnb\" (UID: \"be5e8a2f-2eea-4768-96ad-84dfa11272be\") " pod="kuadrant-system/authorino-65685c7f4d-94cnb" Jun 08 23:01:25.411100 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.411069 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65685c7f4d-94cnb" Jun 08 23:01:25.562764 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.562734 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65685c7f4d-94cnb"] Jun 08 23:01:25.564566 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:01:25.564537 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe5e8a2f_2eea_4768_96ad_84dfa11272be.slice/crio-2b124eb7be923f3d58e699e2dec8aeff824c0ab216847ec753e49248cea79692 WatchSource:0}: Error finding container 2b124eb7be923f3d58e699e2dec8aeff824c0ab216847ec753e49248cea79692: Status 404 returned error can't find the container with id 2b124eb7be923f3d58e699e2dec8aeff824c0ab216847ec753e49248cea79692 Jun 08 23:01:25.936981 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.936951 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hgff9" Jun 08 23:01:25.937165 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.936951 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65685c7f4d-94cnb" event={"ID":"be5e8a2f-2eea-4768-96ad-84dfa11272be","Type":"ContainerStarted","Data":"2b124eb7be923f3d58e699e2dec8aeff824c0ab216847ec753e49248cea79692"} Jun 08 23:01:25.965431 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:25.965377 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hgff9" Jun 08 23:01:26.157688 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.157648 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wkg8\" (UniqueName: \"kubernetes.io/projected/e63b93db-9fb1-42f5-95a6-2e1f2e62fa55-kube-api-access-9wkg8\") pod \"e63b93db-9fb1-42f5-95a6-2e1f2e62fa55\" (UID: \"e63b93db-9fb1-42f5-95a6-2e1f2e62fa55\") " Jun 08 23:01:26.159971 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.159944 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e63b93db-9fb1-42f5-95a6-2e1f2e62fa55-kube-api-access-9wkg8" (OuterVolumeSpecName: "kube-api-access-9wkg8") pod "e63b93db-9fb1-42f5-95a6-2e1f2e62fa55" (UID: "e63b93db-9fb1-42f5-95a6-2e1f2e62fa55"). InnerVolumeSpecName "kube-api-access-9wkg8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:01:26.258514 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.258425 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wkg8\" (UniqueName: \"kubernetes.io/projected/e63b93db-9fb1-42f5-95a6-2e1f2e62fa55-kube-api-access-9wkg8\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:01:26.943230 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.943188 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65685c7f4d-94cnb" event={"ID":"be5e8a2f-2eea-4768-96ad-84dfa11272be","Type":"ContainerStarted","Data":"55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528"} Jun 08 23:01:26.943407 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.943257 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hgff9" Jun 08 23:01:26.956875 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.956827 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65685c7f4d-94cnb" podStartSLOduration=1.539449911 podStartE2EDuration="1.95681458s" totalImagesPullingTime="417.364669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:01:26.955653647 +0000 UTC m=+808.764265502" watchObservedRunningTime="2026-06-08 23:01:26.95681458 +0000 UTC m=+808.765426434" Jun 08 23:01:26.977188 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.977156 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hgff9"] Jun 08 23:01:26.980647 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.980625 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hnmhj"] Jun 08 23:01:26.980797 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.980778 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hnmhj" podUID="9afe1e81-cf0a-4dea-b466-43d768d9af77" containerName="authorino" containerID="cri-o://04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb" gracePeriod=30 Jun 08 23:01:26.984159 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:26.984136 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hgff9"] Jun 08 23:01:27.048529 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.048498 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:01:27.092899 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.092871 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69944d7648-hdsv9"] Jun 08 23:01:27.095110 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.095090 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:01:27.097046 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.096865 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 08 23:01:27.102933 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.102901 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69944d7648-hdsv9"] Jun 08 23:01:27.164007 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.163978 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b295627-7e07-4041-92b4-a2e153287030-tls-cert\") pod \"authorino-69944d7648-hdsv9\" (UID: \"0b295627-7e07-4041-92b4-a2e153287030\") " pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:01:27.164333 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.164041 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qc6c\" (UniqueName: \"kubernetes.io/projected/0b295627-7e07-4041-92b4-a2e153287030-kube-api-access-2qc6c\") pod \"authorino-69944d7648-hdsv9\" (UID: \"0b295627-7e07-4041-92b4-a2e153287030\") " pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:01:27.234739 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.234714 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hnmhj" Jun 08 23:01:27.264850 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.264824 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vd22\" (UniqueName: \"kubernetes.io/projected/9afe1e81-cf0a-4dea-b466-43d768d9af77-kube-api-access-2vd22\") pod \"9afe1e81-cf0a-4dea-b466-43d768d9af77\" (UID: \"9afe1e81-cf0a-4dea-b466-43d768d9af77\") " Jun 08 23:01:27.264991 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.264941 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qc6c\" (UniqueName: \"kubernetes.io/projected/0b295627-7e07-4041-92b4-a2e153287030-kube-api-access-2qc6c\") pod \"authorino-69944d7648-hdsv9\" (UID: \"0b295627-7e07-4041-92b4-a2e153287030\") " pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:01:27.265091 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.264995 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b295627-7e07-4041-92b4-a2e153287030-tls-cert\") pod \"authorino-69944d7648-hdsv9\" (UID: \"0b295627-7e07-4041-92b4-a2e153287030\") " pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:01:27.266891 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.266855 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9afe1e81-cf0a-4dea-b466-43d768d9af77-kube-api-access-2vd22" (OuterVolumeSpecName: "kube-api-access-2vd22") pod "9afe1e81-cf0a-4dea-b466-43d768d9af77" (UID: "9afe1e81-cf0a-4dea-b466-43d768d9af77"). InnerVolumeSpecName "kube-api-access-2vd22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:01:27.267308 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.267290 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b295627-7e07-4041-92b4-a2e153287030-tls-cert\") pod \"authorino-69944d7648-hdsv9\" (UID: \"0b295627-7e07-4041-92b4-a2e153287030\") " pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:01:27.271501 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.271481 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qc6c\" (UniqueName: \"kubernetes.io/projected/0b295627-7e07-4041-92b4-a2e153287030-kube-api-access-2qc6c\") pod \"authorino-69944d7648-hdsv9\" (UID: \"0b295627-7e07-4041-92b4-a2e153287030\") " pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:01:27.365564 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.365530 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vd22\" (UniqueName: \"kubernetes.io/projected/9afe1e81-cf0a-4dea-b466-43d768d9af77-kube-api-access-2vd22\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:01:27.408132 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.408090 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:01:27.534460 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.534434 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69944d7648-hdsv9"] Jun 08 23:01:27.535997 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:01:27.535972 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b295627_7e07_4041_92b4_a2e153287030.slice/crio-85b0c5c30ab8ef9d9c113fca62377b6a6a3d94235b4e80cb0f470b5e6170a0a7 WatchSource:0}: Error finding container 85b0c5c30ab8ef9d9c113fca62377b6a6a3d94235b4e80cb0f470b5e6170a0a7: Status 404 returned error can't find the container with id 85b0c5c30ab8ef9d9c113fca62377b6a6a3d94235b4e80cb0f470b5e6170a0a7 Jun 08 23:01:27.949658 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.949625 2567 generic.go:356] "Generic (PLEG): container finished" podID="9afe1e81-cf0a-4dea-b466-43d768d9af77" containerID="04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb" exitCode=0 Jun 08 23:01:27.949841 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.949674 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hnmhj" Jun 08 23:01:27.949841 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.949673 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hnmhj" event={"ID":"9afe1e81-cf0a-4dea-b466-43d768d9af77","Type":"ContainerDied","Data":"04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb"} Jun 08 23:01:27.949841 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.949799 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hnmhj" event={"ID":"9afe1e81-cf0a-4dea-b466-43d768d9af77","Type":"ContainerDied","Data":"c218946918d3a32c7802dd4c4b5ff5f71afe450d302941f9c8114b863e8c91de"} Jun 08 23:01:27.949841 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.949828 2567 scope.go:117] "RemoveContainer" containerID="04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb" Jun 08 23:01:27.952779 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.952712 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69944d7648-hdsv9" event={"ID":"0b295627-7e07-4041-92b4-a2e153287030","Type":"ContainerStarted","Data":"20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697"} Jun 08 23:01:27.952779 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.952751 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69944d7648-hdsv9" event={"ID":"0b295627-7e07-4041-92b4-a2e153287030","Type":"ContainerStarted","Data":"85b0c5c30ab8ef9d9c113fca62377b6a6a3d94235b4e80cb0f470b5e6170a0a7"} Jun 08 23:01:27.960199 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.960182 2567 scope.go:117] "RemoveContainer" containerID="04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb" Jun 08 23:01:27.960555 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:01:27.960525 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb\": container with ID starting with 04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb not found: ID does not exist" containerID="04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb" Jun 08 23:01:27.960664 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.960559 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb"} err="failed to get container status \"04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb\": rpc error: code = NotFound desc = could not find container \"04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb\": container with ID starting with 04f7319080bd4557fcf19f529cd8cbe9d45303230b43a1728e77a9d806e2ecdb not found: ID does not exist" Jun 08 23:01:27.967367 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.967308 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69944d7648-hdsv9" podStartSLOduration=0.654018418 podStartE2EDuration="967.290576ms" totalImagesPullingTime="313.272158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:01:27.964144722 +0000 UTC m=+809.772756588" watchObservedRunningTime="2026-06-08 23:01:27.967290576 +0000 UTC m=+809.775902433" Jun 08 23:01:27.979256 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.979220 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hnmhj"] Jun 08 23:01:27.988496 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.988454 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hnmhj"] Jun 08 23:01:27.998579 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:27.998541 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65685c7f4d-94cnb"] Jun 08 23:01:28.622179 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:28.622150 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9afe1e81-cf0a-4dea-b466-43d768d9af77" path="/var/lib/kubelet/pods/9afe1e81-cf0a-4dea-b466-43d768d9af77/volumes" Jun 08 23:01:28.622525 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:28.622493 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e63b93db-9fb1-42f5-95a6-2e1f2e62fa55" path="/var/lib/kubelet/pods/e63b93db-9fb1-42f5-95a6-2e1f2e62fa55/volumes" Jun 08 23:01:28.959508 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:28.959473 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-65685c7f4d-94cnb" podUID="be5e8a2f-2eea-4768-96ad-84dfa11272be" containerName="authorino" containerID="cri-o://55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528" gracePeriod=30 Jun 08 23:01:29.209364 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.209338 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65685c7f4d-94cnb" Jun 08 23:01:29.282758 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.282659 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhlls\" (UniqueName: \"kubernetes.io/projected/be5e8a2f-2eea-4768-96ad-84dfa11272be-kube-api-access-qhlls\") pod \"be5e8a2f-2eea-4768-96ad-84dfa11272be\" (UID: \"be5e8a2f-2eea-4768-96ad-84dfa11272be\") " Jun 08 23:01:29.284785 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.284757 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be5e8a2f-2eea-4768-96ad-84dfa11272be-kube-api-access-qhlls" (OuterVolumeSpecName: "kube-api-access-qhlls") pod "be5e8a2f-2eea-4768-96ad-84dfa11272be" (UID: "be5e8a2f-2eea-4768-96ad-84dfa11272be"). InnerVolumeSpecName "kube-api-access-qhlls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:01:29.384081 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.384041 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhlls\" (UniqueName: \"kubernetes.io/projected/be5e8a2f-2eea-4768-96ad-84dfa11272be-kube-api-access-qhlls\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:01:29.965991 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.965957 2567 generic.go:356] "Generic (PLEG): container finished" podID="be5e8a2f-2eea-4768-96ad-84dfa11272be" containerID="55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528" exitCode=0 Jun 08 23:01:29.966445 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.965997 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65685c7f4d-94cnb" event={"ID":"be5e8a2f-2eea-4768-96ad-84dfa11272be","Type":"ContainerDied","Data":"55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528"} Jun 08 23:01:29.966445 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.966021 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65685c7f4d-94cnb" Jun 08 23:01:29.966445 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.966046 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65685c7f4d-94cnb" event={"ID":"be5e8a2f-2eea-4768-96ad-84dfa11272be","Type":"ContainerDied","Data":"2b124eb7be923f3d58e699e2dec8aeff824c0ab216847ec753e49248cea79692"} Jun 08 23:01:29.966445 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.966065 2567 scope.go:117] "RemoveContainer" containerID="55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528" Jun 08 23:01:29.975752 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.975734 2567 scope.go:117] "RemoveContainer" containerID="55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528" Jun 08 23:01:29.976006 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:01:29.975987 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528\": container with ID starting with 55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528 not found: ID does not exist" containerID="55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528" Jun 08 23:01:29.976082 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.976021 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528"} err="failed to get container status \"55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528\": rpc error: code = NotFound desc = could not find container \"55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528\": container with ID starting with 55eab2e3c7818236f60d71b8ef93a1fb9e3b8cf88b07b38bf5de2edd4b16a528 not found: ID does not exist" Jun 08 23:01:29.985458 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.985433 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65685c7f4d-94cnb"] Jun 08 23:01:29.988889 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:29.988866 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-65685c7f4d-94cnb"] Jun 08 23:01:30.419299 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.419275 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:01:30.435684 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.435653 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8f56fd6fb-lc2xl"] Jun 08 23:01:30.435964 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.435951 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afe1e81-cf0a-4dea-b466-43d768d9af77" containerName="authorino" Jun 08 23:01:30.436006 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.435966 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afe1e81-cf0a-4dea-b466-43d768d9af77" containerName="authorino" Jun 08 23:01:30.436006 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.435980 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be5e8a2f-2eea-4768-96ad-84dfa11272be" containerName="authorino" Jun 08 23:01:30.436006 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.435986 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="be5e8a2f-2eea-4768-96ad-84dfa11272be" containerName="authorino" Jun 08 23:01:30.436096 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.436046 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afe1e81-cf0a-4dea-b466-43d768d9af77" containerName="authorino" Jun 08 23:01:30.436096 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.436055 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="be5e8a2f-2eea-4768-96ad-84dfa11272be" containerName="authorino" Jun 08 23:01:30.438519 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.438501 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:30.440364 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.440341 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 23:01:30.440519 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.440503 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 23:01:30.440633 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.440614 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zzhmn\"" Jun 08 23:01:30.446529 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.446510 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8f56fd6fb-lc2xl"] Jun 08 23:01:30.492887 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.492856 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzm67\" (UniqueName: \"kubernetes.io/projected/362c247a-6f5c-4964-b996-ff23fbdfab98-kube-api-access-nzm67\") pod \"maas-controller-8f56fd6fb-lc2xl\" (UID: \"362c247a-6f5c-4964-b996-ff23fbdfab98\") " pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:30.493048 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.492901 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/362c247a-6f5c-4964-b996-ff23fbdfab98-webhook-cert\") pod \"maas-controller-8f56fd6fb-lc2xl\" (UID: \"362c247a-6f5c-4964-b996-ff23fbdfab98\") " pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:30.516347 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.516316 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:01:30.593453 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.593417 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzm67\" (UniqueName: \"kubernetes.io/projected/362c247a-6f5c-4964-b996-ff23fbdfab98-kube-api-access-nzm67\") pod \"maas-controller-8f56fd6fb-lc2xl\" (UID: \"362c247a-6f5c-4964-b996-ff23fbdfab98\") " pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:30.593622 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.593476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/362c247a-6f5c-4964-b996-ff23fbdfab98-webhook-cert\") pod \"maas-controller-8f56fd6fb-lc2xl\" (UID: \"362c247a-6f5c-4964-b996-ff23fbdfab98\") " pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:30.593680 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:01:30.593664 2567 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 08 23:01:30.593754 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:01:30.593743 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/362c247a-6f5c-4964-b996-ff23fbdfab98-webhook-cert podName:362c247a-6f5c-4964-b996-ff23fbdfab98 nodeName:}" failed. No retries permitted until 2026-06-08 23:01:31.093721677 +0000 UTC m=+812.902333510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/362c247a-6f5c-4964-b996-ff23fbdfab98-webhook-cert") pod "maas-controller-8f56fd6fb-lc2xl" (UID: "362c247a-6f5c-4964-b996-ff23fbdfab98") : secret "maas-controller-webhook-cert" not found Jun 08 23:01:30.604176 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.604148 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzm67\" (UniqueName: \"kubernetes.io/projected/362c247a-6f5c-4964-b996-ff23fbdfab98-kube-api-access-nzm67\") pod \"maas-controller-8f56fd6fb-lc2xl\" (UID: \"362c247a-6f5c-4964-b996-ff23fbdfab98\") " pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:30.614878 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.614852 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:01:30.624784 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.624757 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be5e8a2f-2eea-4768-96ad-84dfa11272be" path="/var/lib/kubelet/pods/be5e8a2f-2eea-4768-96ad-84dfa11272be/volumes" Jun 08 23:01:30.625046 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.625032 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84dfc9bfcb-f7f55"] Jun 08 23:01:30.627600 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.627583 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:30.630958 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.630936 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84dfc9bfcb-f7f55"] Jun 08 23:01:30.667648 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.667618 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:01:30.694428 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.694328 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j52r5\" (UniqueName: \"kubernetes.io/projected/3328e962-5566-4bc1-8158-847e730e4e82-kube-api-access-j52r5\") pod \"maas-controller-84dfc9bfcb-f7f55\" (UID: \"3328e962-5566-4bc1-8158-847e730e4e82\") " pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:30.694428 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.694380 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3328e962-5566-4bc1-8158-847e730e4e82-webhook-cert\") pod \"maas-controller-84dfc9bfcb-f7f55\" (UID: \"3328e962-5566-4bc1-8158-847e730e4e82\") " pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:30.795877 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.795840 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j52r5\" (UniqueName: \"kubernetes.io/projected/3328e962-5566-4bc1-8158-847e730e4e82-kube-api-access-j52r5\") pod \"maas-controller-84dfc9bfcb-f7f55\" (UID: \"3328e962-5566-4bc1-8158-847e730e4e82\") " pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:30.796066 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.795900 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3328e962-5566-4bc1-8158-847e730e4e82-webhook-cert\") pod \"maas-controller-84dfc9bfcb-f7f55\" (UID: \"3328e962-5566-4bc1-8158-847e730e4e82\") " pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:30.798371 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.798343 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3328e962-5566-4bc1-8158-847e730e4e82-webhook-cert\") pod \"maas-controller-84dfc9bfcb-f7f55\" (UID: \"3328e962-5566-4bc1-8158-847e730e4e82\") " pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:30.803233 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.803196 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j52r5\" (UniqueName: \"kubernetes.io/projected/3328e962-5566-4bc1-8158-847e730e4e82-kube-api-access-j52r5\") pod \"maas-controller-84dfc9bfcb-f7f55\" (UID: \"3328e962-5566-4bc1-8158-847e730e4e82\") " pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:30.939451 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:30.939415 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:31.071976 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:31.071950 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84dfc9bfcb-f7f55"] Jun 08 23:01:31.073645 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:01:31.073622 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3328e962_5566_4bc1_8158_847e730e4e82.slice/crio-39ae7378ed600b38e0f877d65c6c1cf4e364358adc45432657d30daeca780864 WatchSource:0}: Error finding container 39ae7378ed600b38e0f877d65c6c1cf4e364358adc45432657d30daeca780864: Status 404 returned error can't find the container with id 39ae7378ed600b38e0f877d65c6c1cf4e364358adc45432657d30daeca780864 Jun 08 23:01:31.100157 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:31.100121 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/362c247a-6f5c-4964-b996-ff23fbdfab98-webhook-cert\") pod \"maas-controller-8f56fd6fb-lc2xl\" (UID: \"362c247a-6f5c-4964-b996-ff23fbdfab98\") " pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:31.102442 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:31.102421 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/362c247a-6f5c-4964-b996-ff23fbdfab98-webhook-cert\") pod \"maas-controller-8f56fd6fb-lc2xl\" (UID: \"362c247a-6f5c-4964-b996-ff23fbdfab98\") " pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:31.349557 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:31.349475 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:31.480011 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:31.479981 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8f56fd6fb-lc2xl"] Jun 08 23:01:31.481198 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:01:31.481166 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod362c247a_6f5c_4964_b996_ff23fbdfab98.slice/crio-221e1693af8b377a97b25807611a0a620c87f91317413c5e1e14ba98b6184298 WatchSource:0}: Error finding container 221e1693af8b377a97b25807611a0a620c87f91317413c5e1e14ba98b6184298: Status 404 returned error can't find the container with id 221e1693af8b377a97b25807611a0a620c87f91317413c5e1e14ba98b6184298 Jun 08 23:01:31.986568 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:31.986456 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" event={"ID":"362c247a-6f5c-4964-b996-ff23fbdfab98","Type":"ContainerStarted","Data":"221e1693af8b377a97b25807611a0a620c87f91317413c5e1e14ba98b6184298"} Jun 08 23:01:31.991550 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:31.991475 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" event={"ID":"3328e962-5566-4bc1-8158-847e730e4e82","Type":"ContainerStarted","Data":"39ae7378ed600b38e0f877d65c6c1cf4e364358adc45432657d30daeca780864"} Jun 08 23:01:33.912045 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:33.911999 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:01:34.006411 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:34.006289 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" event={"ID":"3328e962-5566-4bc1-8158-847e730e4e82","Type":"ContainerStarted","Data":"3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5"} Jun 08 23:01:34.006411 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:34.006360 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:34.007874 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:34.007848 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" event={"ID":"362c247a-6f5c-4964-b996-ff23fbdfab98","Type":"ContainerStarted","Data":"40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c"} Jun 08 23:01:34.007993 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:34.007892 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:34.010545 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:34.010519 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:01:34.022511 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:34.022460 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" podStartSLOduration=1.35947077 podStartE2EDuration="4.022443103s" totalImagesPullingTime="2.662972333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:01:34.020565596 +0000 UTC m=+815.829177455" watchObservedRunningTime="2026-06-08 23:01:34.022443103 +0000 UTC m=+815.831054956" Jun 08 23:01:34.037636 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:34.037576 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" podStartSLOduration=1.771549405 podStartE2EDuration="4.03755925s" totalImagesPullingTime="2.266009845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:01:34.035081802 +0000 UTC m=+815.843693657" watchObservedRunningTime="2026-06-08 23:01:34.03755925 +0000 UTC m=+815.846171103" Jun 08 23:01:45.018141 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.018108 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:45.018528 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.018180 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:01:45.070942 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.070906 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8f56fd6fb-lc2xl"] Jun 08 23:01:45.071156 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.071133 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" podUID="362c247a-6f5c-4964-b996-ff23fbdfab98" containerName="manager" containerID="cri-o://40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c" gracePeriod=30 Jun 08 23:01:45.315339 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.315314 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:45.429003 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.428970 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/362c247a-6f5c-4964-b996-ff23fbdfab98-webhook-cert\") pod \"362c247a-6f5c-4964-b996-ff23fbdfab98\" (UID: \"362c247a-6f5c-4964-b996-ff23fbdfab98\") " Jun 08 23:01:45.429198 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.429040 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzm67\" (UniqueName: \"kubernetes.io/projected/362c247a-6f5c-4964-b996-ff23fbdfab98-kube-api-access-nzm67\") pod \"362c247a-6f5c-4964-b996-ff23fbdfab98\" (UID: \"362c247a-6f5c-4964-b996-ff23fbdfab98\") " Jun 08 23:01:45.431228 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.431200 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/362c247a-6f5c-4964-b996-ff23fbdfab98-kube-api-access-nzm67" (OuterVolumeSpecName: "kube-api-access-nzm67") pod "362c247a-6f5c-4964-b996-ff23fbdfab98" (UID: "362c247a-6f5c-4964-b996-ff23fbdfab98"). InnerVolumeSpecName "kube-api-access-nzm67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:01:45.431323 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.431281 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/362c247a-6f5c-4964-b996-ff23fbdfab98-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "362c247a-6f5c-4964-b996-ff23fbdfab98" (UID: "362c247a-6f5c-4964-b996-ff23fbdfab98"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 23:01:45.529748 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.529708 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/362c247a-6f5c-4964-b996-ff23fbdfab98-webhook-cert\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:01:45.529748 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:45.529740 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzm67\" (UniqueName: \"kubernetes.io/projected/362c247a-6f5c-4964-b996-ff23fbdfab98-kube-api-access-nzm67\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:01:46.075199 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.075162 2567 generic.go:356] "Generic (PLEG): container finished" podID="362c247a-6f5c-4964-b996-ff23fbdfab98" containerID="40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c" exitCode=0 Jun 08 23:01:46.075610 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.075214 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" Jun 08 23:01:46.075610 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.075250 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" event={"ID":"362c247a-6f5c-4964-b996-ff23fbdfab98","Type":"ContainerDied","Data":"40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c"} Jun 08 23:01:46.075610 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.075306 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8f56fd6fb-lc2xl" event={"ID":"362c247a-6f5c-4964-b996-ff23fbdfab98","Type":"ContainerDied","Data":"221e1693af8b377a97b25807611a0a620c87f91317413c5e1e14ba98b6184298"} Jun 08 23:01:46.075610 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.075328 2567 scope.go:117] "RemoveContainer" containerID="40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c" Jun 08 23:01:46.084269 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.084248 2567 scope.go:117] "RemoveContainer" containerID="40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c" Jun 08 23:01:46.084532 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:01:46.084511 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c\": container with ID starting with 40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c not found: ID does not exist" containerID="40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c" Jun 08 23:01:46.084584 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.084540 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c"} err="failed to get container status \"40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c\": rpc error: code = NotFound desc = could not find container \"40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c\": container with ID starting with 40564bdc4da5b7dcdb5b216f79d44aac257e3e6555fdac9b09ee2f65f189b58c not found: ID does not exist" Jun 08 23:01:46.095693 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.095666 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8f56fd6fb-lc2xl"] Jun 08 23:01:46.099652 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.099632 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8f56fd6fb-lc2xl"] Jun 08 23:01:46.621817 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.621782 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="362c247a-6f5c-4964-b996-ff23fbdfab98" path="/var/lib/kubelet/pods/362c247a-6f5c-4964-b996-ff23fbdfab98/volumes" Jun 08 23:01:46.798806 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.798755 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:01:46.897672 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:01:46.897595 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:02:05.034830 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.034786 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:02:05.083880 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.083846 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp"] Jun 08 23:02:05.084257 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.084238 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="362c247a-6f5c-4964-b996-ff23fbdfab98" containerName="manager" Jun 08 23:02:05.084351 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.084260 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="362c247a-6f5c-4964-b996-ff23fbdfab98" containerName="manager" Jun 08 23:02:05.084351 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.084343 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="362c247a-6f5c-4964-b996-ff23fbdfab98" containerName="manager" Jun 08 23:02:05.090624 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.089822 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:05.092454 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.092322 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-dockercfg-zmw2z\"" Jun 08 23:02:05.092697 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.092673 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jun 08 23:02:05.095004 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.094977 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp"] Jun 08 23:02:05.203569 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.203537 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8d3b9b1-ddf2-4fd6-a256-b66883caa678-maas-api-tls\") pod \"maas-api-68f7cb4f95-f68dp\" (UID: \"b8d3b9b1-ddf2-4fd6-a256-b66883caa678\") " pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:05.203748 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.203580 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvwfn\" (UniqueName: \"kubernetes.io/projected/b8d3b9b1-ddf2-4fd6-a256-b66883caa678-kube-api-access-rvwfn\") pod \"maas-api-68f7cb4f95-f68dp\" (UID: \"b8d3b9b1-ddf2-4fd6-a256-b66883caa678\") " pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:05.304603 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.304506 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8d3b9b1-ddf2-4fd6-a256-b66883caa678-maas-api-tls\") pod \"maas-api-68f7cb4f95-f68dp\" (UID: \"b8d3b9b1-ddf2-4fd6-a256-b66883caa678\") " pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:05.304603 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.304563 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvwfn\" (UniqueName: \"kubernetes.io/projected/b8d3b9b1-ddf2-4fd6-a256-b66883caa678-kube-api-access-rvwfn\") pod \"maas-api-68f7cb4f95-f68dp\" (UID: \"b8d3b9b1-ddf2-4fd6-a256-b66883caa678\") " pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:05.304893 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:02:05.304692 2567 secret.go:189] Couldn't get secret redhat-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 08 23:02:05.304893 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:02:05.304767 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8d3b9b1-ddf2-4fd6-a256-b66883caa678-maas-api-tls podName:b8d3b9b1-ddf2-4fd6-a256-b66883caa678 nodeName:}" failed. No retries permitted until 2026-06-08 23:02:05.804744903 +0000 UTC m=+847.613356742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8d3b9b1-ddf2-4fd6-a256-b66883caa678-maas-api-tls") pod "maas-api-68f7cb4f95-f68dp" (UID: "b8d3b9b1-ddf2-4fd6-a256-b66883caa678") : secret "maas-api-serving-cert" not found Jun 08 23:02:05.314378 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.314353 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvwfn\" (UniqueName: \"kubernetes.io/projected/b8d3b9b1-ddf2-4fd6-a256-b66883caa678-kube-api-access-rvwfn\") pod \"maas-api-68f7cb4f95-f68dp\" (UID: \"b8d3b9b1-ddf2-4fd6-a256-b66883caa678\") " pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:05.372700 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.372666 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:02:05.476630 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.476596 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:02:05.564558 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.564466 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:02:05.572280 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.572255 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-2xbgw"] Jun 08 23:02:05.575650 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.575629 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" Jun 08 23:02:05.577414 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.577382 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 08 23:02:05.577532 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.577426 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-895pk\"" Jun 08 23:02:05.585134 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.584987 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-2xbgw"] Jun 08 23:02:05.708037 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.708001 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8wfz\" (UniqueName: \"kubernetes.io/projected/db1d4c01-2583-4c42-b29a-02575e863904-kube-api-access-s8wfz\") pod \"payload-pre-processing-67d7689b65-2xbgw\" (UID: \"db1d4c01-2583-4c42-b29a-02575e863904\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" Jun 08 23:02:05.809264 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.809200 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8wfz\" (UniqueName: \"kubernetes.io/projected/db1d4c01-2583-4c42-b29a-02575e863904-kube-api-access-s8wfz\") pod \"payload-pre-processing-67d7689b65-2xbgw\" (UID: \"db1d4c01-2583-4c42-b29a-02575e863904\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" Jun 08 23:02:05.809482 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.809349 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8d3b9b1-ddf2-4fd6-a256-b66883caa678-maas-api-tls\") pod \"maas-api-68f7cb4f95-f68dp\" (UID: \"b8d3b9b1-ddf2-4fd6-a256-b66883caa678\") " pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:05.811803 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.811781 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8d3b9b1-ddf2-4fd6-a256-b66883caa678-maas-api-tls\") pod \"maas-api-68f7cb4f95-f68dp\" (UID: \"b8d3b9b1-ddf2-4fd6-a256-b66883caa678\") " pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:05.816063 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.816007 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8wfz\" (UniqueName: \"kubernetes.io/projected/db1d4c01-2583-4c42-b29a-02575e863904-kube-api-access-s8wfz\") pod \"payload-pre-processing-67d7689b65-2xbgw\" (UID: \"db1d4c01-2583-4c42-b29a-02575e863904\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" Jun 08 23:02:05.888363 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:05.888331 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" Jun 08 23:02:06.001895 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:06.001865 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:06.046248 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:06.046207 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-2xbgw"] Jun 08 23:02:06.047599 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:02:06.047572 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb1d4c01_2583_4c42_b29a_02575e863904.slice/crio-e325c6deb990937a8c94e7c2ba86ed9ad79284ca5ceee54c6feb3ed74cc8de75 WatchSource:0}: Error finding container e325c6deb990937a8c94e7c2ba86ed9ad79284ca5ceee54c6feb3ed74cc8de75: Status 404 returned error can't find the container with id e325c6deb990937a8c94e7c2ba86ed9ad79284ca5ceee54c6feb3ed74cc8de75 Jun 08 23:02:06.145867 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:06.145661 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp"] Jun 08 23:02:06.147967 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:02:06.147940 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d3b9b1_ddf2_4fd6_a256_b66883caa678.slice/crio-315ce8505cafb4b8c03a4eccbaa2e63f5ffd89a9e686c2652573b3c2def5927c WatchSource:0}: Error finding container 315ce8505cafb4b8c03a4eccbaa2e63f5ffd89a9e686c2652573b3c2def5927c: Status 404 returned error can't find the container with id 315ce8505cafb4b8c03a4eccbaa2e63f5ffd89a9e686c2652573b3c2def5927c Jun 08 23:02:06.189099 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:06.188994 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" event={"ID":"b8d3b9b1-ddf2-4fd6-a256-b66883caa678","Type":"ContainerStarted","Data":"315ce8505cafb4b8c03a4eccbaa2e63f5ffd89a9e686c2652573b3c2def5927c"} Jun 08 23:02:06.190958 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:06.190877 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" event={"ID":"db1d4c01-2583-4c42-b29a-02575e863904","Type":"ContainerStarted","Data":"e325c6deb990937a8c94e7c2ba86ed9ad79284ca5ceee54c6feb3ed74cc8de75"} Jun 08 23:02:08.204008 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:08.203971 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" event={"ID":"db1d4c01-2583-4c42-b29a-02575e863904","Type":"ContainerStarted","Data":"8e4d764da188f268b67575919a07cc0b4a3b906b2a47b782738098e51aa871f3"} Jun 08 23:02:08.204408 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:08.204105 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" Jun 08 23:02:08.218383 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:08.218333 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" podStartSLOduration=1.620909829 podStartE2EDuration="3.218317118s" totalImagesPullingTime="1.597407289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:02:08.21665123 +0000 UTC m=+850.025263117" watchObservedRunningTime="2026-06-08 23:02:08.218317118 +0000 UTC m=+850.026928951" Jun 08 23:02:11.224451 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:11.224414 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" event={"ID":"b8d3b9b1-ddf2-4fd6-a256-b66883caa678","Type":"ContainerStarted","Data":"763cf63dfd548c1b678ec64b7d5a491417ec0bf48b09e198e70a650d0acd138f"} Jun 08 23:02:11.224816 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:11.224546 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:11.240587 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:11.240524 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" podStartSLOduration=2.044261335 podStartE2EDuration="6.240507155s" totalImagesPullingTime="4.19624582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:02:11.239328748 +0000 UTC m=+853.047940598" watchObservedRunningTime="2026-06-08 23:02:11.240507155 +0000 UTC m=+853.049119012" Jun 08 23:02:13.236208 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:13.236177 2567 generic.go:356] "Generic (PLEG): container finished" podID="ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6" containerID="b160d3a7c4ac25ff0b5ba418be363f2ba273dc1e39d5fe1b897906172b976b0c" exitCode=0 Jun 08 23:02:13.236689 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:13.236246 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" event={"ID":"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6","Type":"ContainerDied","Data":"b160d3a7c4ac25ff0b5ba418be363f2ba273dc1e39d5fe1b897906172b976b0c"} Jun 08 23:02:13.236689 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:13.236629 2567 scope.go:117] "RemoveContainer" containerID="b160d3a7c4ac25ff0b5ba418be363f2ba273dc1e39d5fe1b897906172b976b0c" Jun 08 23:02:13.237663 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:13.237632 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 23:02:13.237739 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:13.237717 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 23:02:13.237783 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:13.237760 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 23:02:14.247815 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:14.247776 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" event={"ID":"ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6","Type":"ContainerStarted","Data":"2c359e958412c4b07f09f0d59a0c44e2d7c77b1ddb9917101007035ae987f56d"} Jun 08 23:02:14.549215 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:14.549113 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 23:02:14.549805 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:14.549777 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-zggbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jun 08 23:02:14.549999 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:14.549974 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" podUID="ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jun 08 23:02:15.549198 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:15.549142 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-zggbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jun 08 23:02:15.549740 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:15.549220 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" podUID="ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jun 08 23:02:16.548473 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:16.548355 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-zggbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jun 08 23:02:16.548473 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:16.548448 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" podUID="ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jun 08 23:02:17.241783 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:17.241747 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ai-gateway-infra/maas-api-68f7cb4f95-f68dp" Jun 08 23:02:17.549107 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:17.548993 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-zggbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jun 08 23:02:17.549107 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:17.549070 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" podUID="ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jun 08 23:02:18.551887 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:18.551854 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 23:02:18.552246 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:18.552201 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 23:02:18.552902 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:18.552885 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-zggbq" Jun 08 23:02:19.001017 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:19.000983 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 23:02:19.001232 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:19.001192 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" containerID="cri-o://7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733" gracePeriod=30 Jun 08 23:02:19.211902 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:19.211871 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-2xbgw" Jun 08 23:02:21.050232 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.050202 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.155050 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.155013 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdwvq\" (UniqueName: \"kubernetes.io/projected/9726d38f-8d3e-4a49-8461-28a0c0ecb0bf-kube-api-access-vdwvq\") pod \"9726d38f-8d3e-4a49-8461-28a0c0ecb0bf\" (UID: \"9726d38f-8d3e-4a49-8461-28a0c0ecb0bf\") " Jun 08 23:02:21.157191 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.157163 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9726d38f-8d3e-4a49-8461-28a0c0ecb0bf-kube-api-access-vdwvq" (OuterVolumeSpecName: "kube-api-access-vdwvq") pod "9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" (UID: "9726d38f-8d3e-4a49-8461-28a0c0ecb0bf"). InnerVolumeSpecName "kube-api-access-vdwvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:02:21.256571 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.256529 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdwvq\" (UniqueName: \"kubernetes.io/projected/9726d38f-8d3e-4a49-8461-28a0c0ecb0bf-kube-api-access-vdwvq\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:02:21.300572 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.300539 2567 generic.go:356] "Generic (PLEG): container finished" podID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerID="7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733" exitCode=143 Jun 08 23:02:21.300731 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.300589 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.300731 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.300629 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9726d38f-8d3e-4a49-8461-28a0c0ecb0bf","Type":"ContainerDied","Data":"7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733"} Jun 08 23:02:21.300731 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.300684 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9726d38f-8d3e-4a49-8461-28a0c0ecb0bf","Type":"ContainerDied","Data":"dd68ba89eb9697523958da8e4767e10c4608ece143715744270afda6ed617bb6"} Jun 08 23:02:21.300731 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.300705 2567 scope.go:117] "RemoveContainer" containerID="7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733" Jun 08 23:02:21.310633 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.310616 2567 scope.go:117] "RemoveContainer" containerID="7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733" Jun 08 23:02:21.310932 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:02:21.310907 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733\": container with ID starting with 7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733 not found: ID does not exist" containerID="7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733" Jun 08 23:02:21.310980 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.310947 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733"} err="failed to get container status \"7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733\": rpc error: code = NotFound desc = could not find container \"7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733\": container with ID starting with 7d2649516a56b3fa6e94229ed90923c405eec46dec0bcd50441c7113bff8f733 not found: ID does not exist" Jun 08 23:02:21.320911 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.320884 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 23:02:21.323017 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.322990 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 23:02:21.341683 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.341656 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 23:02:21.341985 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.341972 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" Jun 08 23:02:21.342039 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.341987 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" Jun 08 23:02:21.342075 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.342047 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" containerName="keycloak" Jun 08 23:02:21.345860 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.345841 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.347716 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.347697 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 08 23:02:21.347869 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.347754 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 08 23:02:21.347869 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.347771 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 08 23:02:21.347869 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.347777 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-lpxjq\"" Jun 08 23:02:21.347869 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.347803 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 08 23:02:21.353922 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.353901 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 23:02:21.458834 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.458796 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/0742549e-4c94-45f6-b4ed-8a69eddc6a40-test-realms\") pod \"maas-keycloak-0\" (UID: \"0742549e-4c94-45f6-b4ed-8a69eddc6a40\") " pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.458834 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.458845 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b2vp\" (UniqueName: \"kubernetes.io/projected/0742549e-4c94-45f6-b4ed-8a69eddc6a40-kube-api-access-5b2vp\") pod \"maas-keycloak-0\" (UID: \"0742549e-4c94-45f6-b4ed-8a69eddc6a40\") " pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.560320 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.560224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/0742549e-4c94-45f6-b4ed-8a69eddc6a40-test-realms\") pod \"maas-keycloak-0\" (UID: \"0742549e-4c94-45f6-b4ed-8a69eddc6a40\") " pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.560320 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.560271 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b2vp\" (UniqueName: \"kubernetes.io/projected/0742549e-4c94-45f6-b4ed-8a69eddc6a40-kube-api-access-5b2vp\") pod \"maas-keycloak-0\" (UID: \"0742549e-4c94-45f6-b4ed-8a69eddc6a40\") " pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.560954 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.560933 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/0742549e-4c94-45f6-b4ed-8a69eddc6a40-test-realms\") pod \"maas-keycloak-0\" (UID: \"0742549e-4c94-45f6-b4ed-8a69eddc6a40\") " pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.567544 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.567515 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b2vp\" (UniqueName: \"kubernetes.io/projected/0742549e-4c94-45f6-b4ed-8a69eddc6a40-kube-api-access-5b2vp\") pod \"maas-keycloak-0\" (UID: \"0742549e-4c94-45f6-b4ed-8a69eddc6a40\") " pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.656039 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.656006 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:21.787417 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:21.787362 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 23:02:21.788305 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:02:21.788279 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0742549e_4c94_45f6_b4ed_8a69eddc6a40.slice/crio-24bb432da5f6c875b5950211ea17afa86515ffe49c0050599a193bf6c693d08e WatchSource:0}: Error finding container 24bb432da5f6c875b5950211ea17afa86515ffe49c0050599a193bf6c693d08e: Status 404 returned error can't find the container with id 24bb432da5f6c875b5950211ea17afa86515ffe49c0050599a193bf6c693d08e Jun 08 23:02:22.308823 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:22.308552 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0742549e-4c94-45f6-b4ed-8a69eddc6a40","Type":"ContainerStarted","Data":"67af5b6e64d045d9d29c4c19a62a7e8c3bb5adda0e269bddfbe7640fca6bd194"} Jun 08 23:02:22.308823 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:22.308612 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0742549e-4c94-45f6-b4ed-8a69eddc6a40","Type":"ContainerStarted","Data":"24bb432da5f6c875b5950211ea17afa86515ffe49c0050599a193bf6c693d08e"} Jun 08 23:02:22.323759 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:22.323494 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.009390125 podStartE2EDuration="1.323473802s" totalImagesPullingTime="314.083677ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:02:22.322890266 +0000 UTC m=+864.131502121" watchObservedRunningTime="2026-06-08 23:02:22.323473802 +0000 UTC m=+864.132085660" Jun 08 23:02:22.624583 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:22.624543 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9726d38f-8d3e-4a49-8461-28a0c0ecb0bf" path="/var/lib/kubelet/pods/9726d38f-8d3e-4a49-8461-28a0c0ecb0bf/volumes" Jun 08 23:02:22.656869 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:22.656826 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:22.659115 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:22.659068 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:23.656621 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:23.656563 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:24.657495 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:24.657438 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:25.657305 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:25.657254 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:26.657265 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:26.657222 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:27.657280 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:27.657226 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:28.656708 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:28.656663 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:29.657360 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:29.657302 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:30.657385 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:30.657328 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:31.656943 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:31.656893 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:31.657295 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:31.657243 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:32.657454 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:32.657386 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:33.657157 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:33.657090 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:34.656492 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:34.656452 2567 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.49:9000/health/started\": dial tcp 10.134.0.49:9000: connect: connection refused" Jun 08 23:02:35.751792 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:35.751754 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:35.769767 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:35.769724 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="0742549e-4c94-45f6-b4ed-8a69eddc6a40" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 23:02:45.757474 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:45.757364 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 08 23:02:56.481819 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.481777 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b49dc6db-dpjvf"] Jun 08 23:02:56.486108 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.486085 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.488620 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.488601 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 08 23:02:56.506100 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.506074 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b49dc6db-dpjvf"] Jun 08 23:02:56.593306 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.593273 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snwlx\" (UniqueName: \"kubernetes.io/projected/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-kube-api-access-snwlx\") pod \"authorino-8b49dc6db-dpjvf\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.593306 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.593310 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-tls-cert\") pod \"authorino-8b49dc6db-dpjvf\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.593560 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.593421 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-oidc-ca\") pod \"authorino-8b49dc6db-dpjvf\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.694544 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.694500 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snwlx\" (UniqueName: \"kubernetes.io/projected/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-kube-api-access-snwlx\") pod \"authorino-8b49dc6db-dpjvf\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.694544 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.694546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-tls-cert\") pod \"authorino-8b49dc6db-dpjvf\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.694753 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.694590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-oidc-ca\") pod \"authorino-8b49dc6db-dpjvf\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.695256 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.695231 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-oidc-ca\") pod \"authorino-8b49dc6db-dpjvf\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.697034 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.697001 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-tls-cert\") pod \"authorino-8b49dc6db-dpjvf\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.701430 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.701411 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snwlx\" (UniqueName: \"kubernetes.io/projected/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-kube-api-access-snwlx\") pod \"authorino-8b49dc6db-dpjvf\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.795044 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.794962 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:02:56.921663 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:56.921635 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b49dc6db-dpjvf"] Jun 08 23:02:56.923112 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:02:56.923086 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb00cb9e_b8dd_4577_a149_0a7a1ac447c6.slice/crio-44b48a387ef4544c6b55dafca659f4ad797d80780373f0eca63f5c08ed02dd58 WatchSource:0}: Error finding container 44b48a387ef4544c6b55dafca659f4ad797d80780373f0eca63f5c08ed02dd58: Status 404 returned error can't find the container with id 44b48a387ef4544c6b55dafca659f4ad797d80780373f0eca63f5c08ed02dd58 Jun 08 23:02:57.547985 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:57.547942 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" event={"ID":"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6","Type":"ContainerStarted","Data":"f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30"} Jun 08 23:02:57.548358 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:57.547997 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" event={"ID":"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6","Type":"ContainerStarted","Data":"44b48a387ef4544c6b55dafca659f4ad797d80780373f0eca63f5c08ed02dd58"} Jun 08 23:02:57.563876 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:57.563809 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" podStartSLOduration=1.1666287770000001 podStartE2EDuration="1.563786157s" totalImagesPullingTime="397.15738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:02:57.563044557 +0000 UTC m=+899.371656415" watchObservedRunningTime="2026-06-08 23:02:57.563786157 +0000 UTC m=+899.372398012" Jun 08 23:02:57.589548 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:57.588756 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69944d7648-hdsv9"] Jun 08 23:02:57.589548 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:57.588969 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-69944d7648-hdsv9" podUID="0b295627-7e07-4041-92b4-a2e153287030" containerName="authorino" containerID="cri-o://20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697" gracePeriod=30 Jun 08 23:02:57.869156 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:57.869121 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:02:58.003053 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.003012 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:02:58.007058 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.007033 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qc6c\" (UniqueName: \"kubernetes.io/projected/0b295627-7e07-4041-92b4-a2e153287030-kube-api-access-2qc6c\") pod \"0b295627-7e07-4041-92b4-a2e153287030\" (UID: \"0b295627-7e07-4041-92b4-a2e153287030\") " Jun 08 23:02:58.007195 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.007143 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b295627-7e07-4041-92b4-a2e153287030-tls-cert\") pod \"0b295627-7e07-4041-92b4-a2e153287030\" (UID: \"0b295627-7e07-4041-92b4-a2e153287030\") " Jun 08 23:02:58.009133 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.009103 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b295627-7e07-4041-92b4-a2e153287030-kube-api-access-2qc6c" (OuterVolumeSpecName: "kube-api-access-2qc6c") pod "0b295627-7e07-4041-92b4-a2e153287030" (UID: "0b295627-7e07-4041-92b4-a2e153287030"). InnerVolumeSpecName "kube-api-access-2qc6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:02:58.016963 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.016941 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b295627-7e07-4041-92b4-a2e153287030-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0b295627-7e07-4041-92b4-a2e153287030" (UID: "0b295627-7e07-4041-92b4-a2e153287030"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 23:02:58.089208 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.089176 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:02:58.108156 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.108072 2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b295627-7e07-4041-92b4-a2e153287030-tls-cert\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:02:58.108156 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.108104 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qc6c\" (UniqueName: \"kubernetes.io/projected/0b295627-7e07-4041-92b4-a2e153287030-kube-api-access-2qc6c\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:02:58.554347 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.554314 2567 generic.go:356] "Generic (PLEG): container finished" podID="0b295627-7e07-4041-92b4-a2e153287030" containerID="20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697" exitCode=0 Jun 08 23:02:58.554763 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.554365 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69944d7648-hdsv9" Jun 08 23:02:58.554763 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.554411 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69944d7648-hdsv9" event={"ID":"0b295627-7e07-4041-92b4-a2e153287030","Type":"ContainerDied","Data":"20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697"} Jun 08 23:02:58.554763 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.554453 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69944d7648-hdsv9" event={"ID":"0b295627-7e07-4041-92b4-a2e153287030","Type":"ContainerDied","Data":"85b0c5c30ab8ef9d9c113fca62377b6a6a3d94235b4e80cb0f470b5e6170a0a7"} Jun 08 23:02:58.554763 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.554476 2567 scope.go:117] "RemoveContainer" containerID="20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697" Jun 08 23:02:58.564948 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.564923 2567 scope.go:117] "RemoveContainer" containerID="20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697" Jun 08 23:02:58.565247 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:02:58.565226 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697\": container with ID starting with 20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697 not found: ID does not exist" containerID="20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697" Jun 08 23:02:58.565319 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.565260 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697"} err="failed to get container status \"20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697\": rpc error: code = NotFound desc = could not find container \"20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697\": container with ID starting with 20346074c58e8dc0a6d9ece2a5cdd8a8fc482e5809240e4e9c1cc1f475b39697 not found: ID does not exist" Jun 08 23:02:58.574877 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.574849 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69944d7648-hdsv9"] Jun 08 23:02:58.580845 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.580821 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69944d7648-hdsv9"] Jun 08 23:02:58.622654 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.622623 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b295627-7e07-4041-92b4-a2e153287030" path="/var/lib/kubelet/pods/0b295627-7e07-4041-92b4-a2e153287030/volumes" Jun 08 23:02:58.638113 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.638087 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 23:02:58.641453 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.641432 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 23:02:58.643093 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.643074 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 23:02:58.646329 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:02:58.646311 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 23:03:11.036024 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.035985 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm"] Jun 08 23:03:11.036556 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.036380 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b295627-7e07-4041-92b4-a2e153287030" containerName="authorino" Jun 08 23:03:11.036556 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.036412 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b295627-7e07-4041-92b4-a2e153287030" containerName="authorino" Jun 08 23:03:11.036556 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.036502 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b295627-7e07-4041-92b4-a2e153287030" containerName="authorino" Jun 08 23:03:11.039586 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.039568 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.042140 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.042114 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kdf8k\"" Jun 08 23:03:11.042261 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.042140 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 08 23:03:11.042261 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.042200 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 08 23:03:11.042261 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.042250 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 08 23:03:11.047237 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.047215 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm"] Jun 08 23:03:11.109014 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.108979 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:03:11.121208 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.121181 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.121334 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.121219 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxtrs\" (UniqueName: \"kubernetes.io/projected/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-kube-api-access-hxtrs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.121334 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.121244 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.121475 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.121371 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.121475 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.121467 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.121562 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.121490 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.222729 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.222667 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.222923 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.222755 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.222923 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.222780 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.222923 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.222822 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.222923 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.222855 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxtrs\" (UniqueName: \"kubernetes.io/projected/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-kube-api-access-hxtrs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.222923 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.222886 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.223217 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.223106 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.223217 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.223170 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.223217 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.223191 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.225071 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.225050 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.225342 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.225321 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.229510 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.229487 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxtrs\" (UniqueName: \"kubernetes.io/projected/f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed-kube-api-access-hxtrs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm\" (UID: \"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.350306 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.350211 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:11.484600 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.484564 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm"] Jun 08 23:03:11.485721 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:03:11.485698 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00e5cbf_2fe1_451e_9b6d_8e567c11e5ed.slice/crio-a58c421f18191d4e01cba9439f9df7acbaaf700432b4a62317523eb4c383338e WatchSource:0}: Error finding container a58c421f18191d4e01cba9439f9df7acbaaf700432b4a62317523eb4c383338e: Status 404 returned error can't find the container with id a58c421f18191d4e01cba9439f9df7acbaaf700432b4a62317523eb4c383338e Jun 08 23:03:11.626740 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:11.626649 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" event={"ID":"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed","Type":"ContainerStarted","Data":"a58c421f18191d4e01cba9439f9df7acbaaf700432b4a62317523eb4c383338e"} Jun 08 23:03:12.519122 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:12.519089 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:03:12.569116 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:12.569083 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:03:15.659219 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:15.659179 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" event={"ID":"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed","Type":"ContainerStarted","Data":"34c0edc53b95695273d680bcc5219393a8154129b763221edf624be59e656e9c"} Jun 08 23:03:23.706219 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:23.706185 2567 generic.go:356] "Generic (PLEG): container finished" podID="f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed" containerID="34c0edc53b95695273d680bcc5219393a8154129b763221edf624be59e656e9c" exitCode=0 Jun 08 23:03:23.706615 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:23.706236 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" event={"ID":"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed","Type":"ContainerDied","Data":"34c0edc53b95695273d680bcc5219393a8154129b763221edf624be59e656e9c"} Jun 08 23:03:23.942627 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:23.942592 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:03:25.720331 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:25.720292 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" event={"ID":"f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed","Type":"ContainerStarted","Data":"5a0c02354318c41e400411521182a384421f788d5f1fe9738e1707075e3af0a8"} Jun 08 23:03:25.736303 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:25.736232 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" podStartSLOduration=1.845745812 podStartE2EDuration="14.736212838s" totalImagesPullingTime="4.804004726s" totalInitContainerRuntime="8.0864623s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:03:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:03:25.735834753 +0000 UTC m=+927.544446606" watchObservedRunningTime="2026-06-08 23:03:25.736212838 +0000 UTC m=+927.544824703" Jun 08 23:03:31.350989 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:31.350950 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:31.351472 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:31.351052 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:31.363820 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:31.363795 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:31.762270 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:31.762237 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm" Jun 08 23:03:39.629729 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.629689 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7"] Jun 08 23:03:39.651738 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.651702 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7"] Jun 08 23:03:39.651888 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.651818 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.654124 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.654102 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 08 23:03:39.657443 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.657419 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:03:39.781792 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.781756 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.781792 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.781795 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.782060 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.781818 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.782060 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.781929 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.782060 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.781977 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2sxt\" (UniqueName: \"kubernetes.io/projected/b6c2b717-006e-496c-b20f-6380fdd563e1-kube-api-access-k2sxt\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.782184 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.782081 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6c2b717-006e-496c-b20f-6380fdd563e1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.883038 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.882928 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.883038 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.882973 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.883038 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.883027 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.883321 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.883070 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.883321 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.883105 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2sxt\" (UniqueName: \"kubernetes.io/projected/b6c2b717-006e-496c-b20f-6380fdd563e1-kube-api-access-k2sxt\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.883321 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.883208 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6c2b717-006e-496c-b20f-6380fdd563e1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.883523 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.883424 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.883578 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.883521 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.883636 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.883596 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.885344 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.885314 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6c2b717-006e-496c-b20f-6380fdd563e1-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.885589 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.885574 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6c2b717-006e-496c-b20f-6380fdd563e1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.889515 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.889497 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2sxt\" (UniqueName: \"kubernetes.io/projected/b6c2b717-006e-496c-b20f-6380fdd563e1-kube-api-access-k2sxt\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7\" (UID: \"b6c2b717-006e-496c-b20f-6380fdd563e1\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:39.962619 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:39.962584 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:40.099238 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:40.099205 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7"] Jun 08 23:03:40.101434 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:03:40.101386 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6c2b717_006e_496c_b20f_6380fdd563e1.slice/crio-7736cfba23a380265196acb648bf851127b95ea6e01109ff0a8726cdc5a820d3 WatchSource:0}: Error finding container 7736cfba23a380265196acb648bf851127b95ea6e01109ff0a8726cdc5a820d3: Status 404 returned error can't find the container with id 7736cfba23a380265196acb648bf851127b95ea6e01109ff0a8726cdc5a820d3 Jun 08 23:03:40.797196 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:40.797160 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" event={"ID":"b6c2b717-006e-496c-b20f-6380fdd563e1","Type":"ContainerStarted","Data":"7cb8ab7bfb42bb04827a2e96341704623e18cf40f2bb3af8aec218535bf5ad7c"} Jun 08 23:03:40.797596 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:40.797214 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" event={"ID":"b6c2b717-006e-496c-b20f-6380fdd563e1","Type":"ContainerStarted","Data":"7736cfba23a380265196acb648bf851127b95ea6e01109ff0a8726cdc5a820d3"} Jun 08 23:03:41.804665 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:41.804573 2567 generic.go:356] "Generic (PLEG): container finished" podID="b6c2b717-006e-496c-b20f-6380fdd563e1" containerID="7cb8ab7bfb42bb04827a2e96341704623e18cf40f2bb3af8aec218535bf5ad7c" exitCode=0 Jun 08 23:03:41.804665 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:41.804649 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" event={"ID":"b6c2b717-006e-496c-b20f-6380fdd563e1","Type":"ContainerDied","Data":"7cb8ab7bfb42bb04827a2e96341704623e18cf40f2bb3af8aec218535bf5ad7c"} Jun 08 23:03:42.811968 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:42.811924 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" event={"ID":"b6c2b717-006e-496c-b20f-6380fdd563e1","Type":"ContainerStarted","Data":"c959a76faa6bb8a4d95f256bf58154bacecb7c108e92c1a197070e4db253a0e0"} Jun 08 23:03:42.831121 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:42.831045 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" podStartSLOduration=2.304603254 podStartE2EDuration="3.831030577s" totalImagesPullingTime="204.157764ms" totalInitContainerRuntime="1.322269559s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:03:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:03:42.829993724 +0000 UTC m=+944.638605576" watchObservedRunningTime="2026-06-08 23:03:42.831030577 +0000 UTC m=+944.639642427" Jun 08 23:03:46.536682 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:46.536649 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:03:49.962945 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:49.962909 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:49.963366 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:49.963063 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:49.975973 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:49.975950 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:50.867573 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:50.867544 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7" Jun 08 23:03:54.246562 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:54.246527 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:03:57.914898 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.914856 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9"] Jun 08 23:03:57.921467 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.921443 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:57.923650 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.923629 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 08 23:03:57.929272 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.929248 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9"] Jun 08 23:03:57.940558 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.940529 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:03:57.944421 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.944380 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:57.944513 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.944433 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:57.944513 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.944455 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:57.944513 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.944483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ae95d94-353c-4e59-a01b-cf3a4da400e3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:57.944610 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.944577 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm5bv\" (UniqueName: \"kubernetes.io/projected/4ae95d94-353c-4e59-a01b-cf3a4da400e3-kube-api-access-sm5bv\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:57.944647 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:57.944629 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.045802 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.045767 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm5bv\" (UniqueName: \"kubernetes.io/projected/4ae95d94-353c-4e59-a01b-cf3a4da400e3-kube-api-access-sm5bv\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.046008 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.045825 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.046008 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.045863 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.046008 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.045887 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.046008 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.045904 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.046008 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.045941 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ae95d94-353c-4e59-a01b-cf3a4da400e3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.046325 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.046300 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.046437 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.046380 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.046482 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.046435 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.048305 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.048282 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ae95d94-353c-4e59-a01b-cf3a4da400e3-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.048482 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.048465 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ae95d94-353c-4e59-a01b-cf3a4da400e3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.054798 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.054772 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm5bv\" (UniqueName: \"kubernetes.io/projected/4ae95d94-353c-4e59-a01b-cf3a4da400e3-kube-api-access-sm5bv\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-ksnm9\" (UID: \"4ae95d94-353c-4e59-a01b-cf3a4da400e3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.231219 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.231167 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:03:58.372330 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.372296 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9"] Jun 08 23:03:58.373927 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:03:58.373905 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae95d94_353c_4e59_a01b_cf3a4da400e3.slice/crio-1e8312638e12da1b950a0acdcd0c7c7ae9ff953dfd111eb20f7fd8bcbcc6952c WatchSource:0}: Error finding container 1e8312638e12da1b950a0acdcd0c7c7ae9ff953dfd111eb20f7fd8bcbcc6952c: Status 404 returned error can't find the container with id 1e8312638e12da1b950a0acdcd0c7c7ae9ff953dfd111eb20f7fd8bcbcc6952c Jun 08 23:03:58.904254 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.904206 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" event={"ID":"4ae95d94-353c-4e59-a01b-cf3a4da400e3","Type":"ContainerStarted","Data":"e39a66c8eb60889e16cb92b5343f1166bd4037d722213109fca8344bf5b36583"} Jun 08 23:03:58.904467 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:58.904264 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" event={"ID":"4ae95d94-353c-4e59-a01b-cf3a4da400e3","Type":"ContainerStarted","Data":"1e8312638e12da1b950a0acdcd0c7c7ae9ff953dfd111eb20f7fd8bcbcc6952c"} Jun 08 23:03:59.910810 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:59.910775 2567 generic.go:356] "Generic (PLEG): container finished" podID="4ae95d94-353c-4e59-a01b-cf3a4da400e3" containerID="e39a66c8eb60889e16cb92b5343f1166bd4037d722213109fca8344bf5b36583" exitCode=0 Jun 08 23:03:59.911192 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:59.910861 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" event={"ID":"4ae95d94-353c-4e59-a01b-cf3a4da400e3","Type":"ContainerDied","Data":"e39a66c8eb60889e16cb92b5343f1166bd4037d722213109fca8344bf5b36583"} Jun 08 23:03:59.911361 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:03:59.911346 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 23:04:00.918328 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:00.918293 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" event={"ID":"4ae95d94-353c-4e59-a01b-cf3a4da400e3","Type":"ContainerStarted","Data":"11ff4b11272e7a2c0aa573d4c41136a71c197c59af55ed289f97119914492f6b"} Jun 08 23:04:00.950829 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:00.950760 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" podStartSLOduration=2.400868855 podStartE2EDuration="3.950737495s" totalImagesPullingTime="205.982628ms" totalInitContainerRuntime="1.343886012s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:04:00.947633234 +0000 UTC m=+962.756245089" watchObservedRunningTime="2026-06-08 23:04:00.950737495 +0000 UTC m=+962.759349349" Jun 08 23:04:08.231759 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:08.231671 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:04:08.231759 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:08.231736 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:04:08.244295 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:08.244270 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:04:08.973445 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:08.973408 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-ksnm9" Jun 08 23:04:53.093742 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.093706 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59d588cc4c-r82ck"] Jun 08 23:04:53.097108 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.097091 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.102857 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.102831 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59d588cc4c-r82ck"] Jun 08 23:04:53.151806 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.151775 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/f38d6321-c41d-43f5-8ef5-4f3dac3899a0-oidc-ca\") pod \"authorino-59d588cc4c-r82ck\" (UID: \"f38d6321-c41d-43f5-8ef5-4f3dac3899a0\") " pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.151987 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.151906 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f38d6321-c41d-43f5-8ef5-4f3dac3899a0-tls-cert\") pod \"authorino-59d588cc4c-r82ck\" (UID: \"f38d6321-c41d-43f5-8ef5-4f3dac3899a0\") " pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.151987 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.151943 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbldf\" (UniqueName: \"kubernetes.io/projected/f38d6321-c41d-43f5-8ef5-4f3dac3899a0-kube-api-access-jbldf\") pod \"authorino-59d588cc4c-r82ck\" (UID: \"f38d6321-c41d-43f5-8ef5-4f3dac3899a0\") " pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.253517 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.253479 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f38d6321-c41d-43f5-8ef5-4f3dac3899a0-tls-cert\") pod \"authorino-59d588cc4c-r82ck\" (UID: \"f38d6321-c41d-43f5-8ef5-4f3dac3899a0\") " pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.253517 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.253519 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbldf\" (UniqueName: \"kubernetes.io/projected/f38d6321-c41d-43f5-8ef5-4f3dac3899a0-kube-api-access-jbldf\") pod \"authorino-59d588cc4c-r82ck\" (UID: \"f38d6321-c41d-43f5-8ef5-4f3dac3899a0\") " pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.253722 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.253588 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/f38d6321-c41d-43f5-8ef5-4f3dac3899a0-oidc-ca\") pod \"authorino-59d588cc4c-r82ck\" (UID: \"f38d6321-c41d-43f5-8ef5-4f3dac3899a0\") " pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.254358 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.254330 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/f38d6321-c41d-43f5-8ef5-4f3dac3899a0-oidc-ca\") pod \"authorino-59d588cc4c-r82ck\" (UID: \"f38d6321-c41d-43f5-8ef5-4f3dac3899a0\") " pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.256322 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.256298 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f38d6321-c41d-43f5-8ef5-4f3dac3899a0-tls-cert\") pod \"authorino-59d588cc4c-r82ck\" (UID: \"f38d6321-c41d-43f5-8ef5-4f3dac3899a0\") " pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.260170 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.260147 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbldf\" (UniqueName: \"kubernetes.io/projected/f38d6321-c41d-43f5-8ef5-4f3dac3899a0-kube-api-access-jbldf\") pod \"authorino-59d588cc4c-r82ck\" (UID: \"f38d6321-c41d-43f5-8ef5-4f3dac3899a0\") " pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.407024 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.406933 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59d588cc4c-r82ck" Jun 08 23:04:53.538474 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:53.538448 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59d588cc4c-r82ck"] Jun 08 23:04:53.540121 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:04:53.540089 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf38d6321_c41d_43f5_8ef5_4f3dac3899a0.slice/crio-ceabd1adf753a22cd2272595d55ec72733d6d6cfe7e7d19c247fa3cd89dd7a56 WatchSource:0}: Error finding container ceabd1adf753a22cd2272595d55ec72733d6d6cfe7e7d19c247fa3cd89dd7a56: Status 404 returned error can't find the container with id ceabd1adf753a22cd2272595d55ec72733d6d6cfe7e7d19c247fa3cd89dd7a56 Jun 08 23:04:54.218174 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.218069 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59d588cc4c-r82ck" event={"ID":"f38d6321-c41d-43f5-8ef5-4f3dac3899a0","Type":"ContainerStarted","Data":"2aa764ca8f4f1b91bd437ad161e59851c027a102b89155aadffef3ae5f223808"} Jun 08 23:04:54.218174 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.218136 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59d588cc4c-r82ck" event={"ID":"f38d6321-c41d-43f5-8ef5-4f3dac3899a0","Type":"ContainerStarted","Data":"ceabd1adf753a22cd2272595d55ec72733d6d6cfe7e7d19c247fa3cd89dd7a56"} Jun 08 23:04:54.234848 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.234781 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59d588cc4c-r82ck" podStartSLOduration=0.850384108 podStartE2EDuration="1.234760443s" totalImagesPullingTime="384.376335ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:04:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:04:54.232924114 +0000 UTC m=+1016.041535967" watchObservedRunningTime="2026-06-08 23:04:54.234760443 +0000 UTC m=+1016.043372299" Jun 08 23:04:54.278125 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.278091 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b49dc6db-dpjvf"] Jun 08 23:04:54.278313 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.278279 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" podUID="cb00cb9e-b8dd-4577-a149-0a7a1ac447c6" containerName="authorino" containerID="cri-o://f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30" gracePeriod=30 Jun 08 23:04:54.538949 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.538923 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:04:54.666940 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.666908 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snwlx\" (UniqueName: \"kubernetes.io/projected/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-kube-api-access-snwlx\") pod \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " Jun 08 23:04:54.667156 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.667011 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-tls-cert\") pod \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " Jun 08 23:04:54.667156 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.667062 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-oidc-ca\") pod \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\" (UID: \"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6\") " Jun 08 23:04:54.669069 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.669032 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-kube-api-access-snwlx" (OuterVolumeSpecName: "kube-api-access-snwlx") pod "cb00cb9e-b8dd-4577-a149-0a7a1ac447c6" (UID: "cb00cb9e-b8dd-4577-a149-0a7a1ac447c6"). InnerVolumeSpecName "kube-api-access-snwlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:04:54.671777 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.671753 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "cb00cb9e-b8dd-4577-a149-0a7a1ac447c6" (UID: "cb00cb9e-b8dd-4577-a149-0a7a1ac447c6"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 23:04:54.678542 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.678513 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "cb00cb9e-b8dd-4577-a149-0a7a1ac447c6" (UID: "cb00cb9e-b8dd-4577-a149-0a7a1ac447c6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 23:04:54.768196 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.768103 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snwlx\" (UniqueName: \"kubernetes.io/projected/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-kube-api-access-snwlx\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:04:54.768196 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.768136 2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-tls-cert\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:04:54.768196 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.768146 2567 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6-oidc-ca\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:04:54.911435 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:54.911378 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:04:55.009979 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.009926 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:04:55.224821 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.224788 2567 generic.go:356] "Generic (PLEG): container finished" podID="cb00cb9e-b8dd-4577-a149-0a7a1ac447c6" containerID="f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30" exitCode=0 Jun 08 23:04:55.225220 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.224832 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" Jun 08 23:04:55.225220 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.224877 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" event={"ID":"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6","Type":"ContainerDied","Data":"f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30"} Jun 08 23:04:55.225220 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.224916 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b49dc6db-dpjvf" event={"ID":"cb00cb9e-b8dd-4577-a149-0a7a1ac447c6","Type":"ContainerDied","Data":"44b48a387ef4544c6b55dafca659f4ad797d80780373f0eca63f5c08ed02dd58"} Jun 08 23:04:55.225220 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.224942 2567 scope.go:117] "RemoveContainer" containerID="f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30" Jun 08 23:04:55.235981 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.235961 2567 scope.go:117] "RemoveContainer" containerID="f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30" Jun 08 23:04:55.236231 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:04:55.236214 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30\": container with ID starting with f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30 not found: ID does not exist" containerID="f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30" Jun 08 23:04:55.236273 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.236239 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30"} err="failed to get container status \"f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30\": rpc error: code = NotFound desc = could not find container \"f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30\": container with ID starting with f8c1013adfc80a86712b15f16afa5ab0ccd886b326c31637c083c09336e20a30 not found: ID does not exist" Jun 08 23:04:55.247498 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.247475 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b49dc6db-dpjvf"] Jun 08 23:04:55.249733 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.249712 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b49dc6db-dpjvf"] Jun 08 23:04:55.985057 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:55.985021 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:04:56.084211 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:56.084182 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:04:56.621334 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:04:56.621302 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb00cb9e-b8dd-4577-a149-0a7a1ac447c6" path="/var/lib/kubelet/pods/cb00cb9e-b8dd-4577-a149-0a7a1ac447c6/volumes" Jun 08 23:06:21.576131 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:21.576098 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:06:21.679184 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:21.679148 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84dfc9bfcb-f7f55"] Jun 08 23:06:21.679377 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:21.679339 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" podUID="3328e962-5566-4bc1-8158-847e730e4e82" containerName="manager" containerID="cri-o://3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5" gracePeriod=30 Jun 08 23:06:21.931986 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:21.931961 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:06:21.963408 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:21.963359 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3328e962-5566-4bc1-8158-847e730e4e82-webhook-cert\") pod \"3328e962-5566-4bc1-8158-847e730e4e82\" (UID: \"3328e962-5566-4bc1-8158-847e730e4e82\") " Jun 08 23:06:21.963587 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:21.963442 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j52r5\" (UniqueName: \"kubernetes.io/projected/3328e962-5566-4bc1-8158-847e730e4e82-kube-api-access-j52r5\") pod \"3328e962-5566-4bc1-8158-847e730e4e82\" (UID: \"3328e962-5566-4bc1-8158-847e730e4e82\") " Jun 08 23:06:21.965624 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:21.965586 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3328e962-5566-4bc1-8158-847e730e4e82-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3328e962-5566-4bc1-8158-847e730e4e82" (UID: "3328e962-5566-4bc1-8158-847e730e4e82"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 23:06:21.965768 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:21.965589 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3328e962-5566-4bc1-8158-847e730e4e82-kube-api-access-j52r5" (OuterVolumeSpecName: "kube-api-access-j52r5") pod "3328e962-5566-4bc1-8158-847e730e4e82" (UID: "3328e962-5566-4bc1-8158-847e730e4e82"). InnerVolumeSpecName "kube-api-access-j52r5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 23:06:22.064126 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.064078 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3328e962-5566-4bc1-8158-847e730e4e82-webhook-cert\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:06:22.064126 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.064120 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j52r5\" (UniqueName: \"kubernetes.io/projected/3328e962-5566-4bc1-8158-847e730e4e82-kube-api-access-j52r5\") on node \"ip-10-0-138-248.ec2.internal\" DevicePath \"\"" Jun 08 23:06:22.708302 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.708267 2567 generic.go:356] "Generic (PLEG): container finished" podID="3328e962-5566-4bc1-8158-847e730e4e82" containerID="3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5" exitCode=0 Jun 08 23:06:22.708733 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.708311 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" Jun 08 23:06:22.708733 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.708349 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" event={"ID":"3328e962-5566-4bc1-8158-847e730e4e82","Type":"ContainerDied","Data":"3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5"} Jun 08 23:06:22.708733 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.708409 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84dfc9bfcb-f7f55" event={"ID":"3328e962-5566-4bc1-8158-847e730e4e82","Type":"ContainerDied","Data":"39ae7378ed600b38e0f877d65c6c1cf4e364358adc45432657d30daeca780864"} Jun 08 23:06:22.708733 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.708435 2567 scope.go:117] "RemoveContainer" containerID="3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5" Jun 08 23:06:22.717642 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.717622 2567 scope.go:117] "RemoveContainer" containerID="3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5" Jun 08 23:06:22.717896 ip-10-0-138-248 kubenswrapper[2567]: E0608 23:06:22.717878 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5\": container with ID starting with 3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5 not found: ID does not exist" containerID="3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5" Jun 08 23:06:22.717946 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.717904 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5"} err="failed to get container status \"3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5\": rpc error: code = NotFound desc = could not find container \"3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5\": container with ID starting with 3564234ad25ff2932429afbe376cb2e53aaa53042c77c4124e13ef1f8afa8ae5 not found: ID does not exist" Jun 08 23:06:22.723962 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.723938 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84dfc9bfcb-f7f55"] Jun 08 23:06:22.727650 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:22.727625 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84dfc9bfcb-f7f55"] Jun 08 23:06:23.353309 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.353279 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:06:23.465014 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.464978 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84dfc9bfcb-mxrsv"] Jun 08 23:06:23.465321 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.465310 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb00cb9e-b8dd-4577-a149-0a7a1ac447c6" containerName="authorino" Jun 08 23:06:23.465365 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.465323 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb00cb9e-b8dd-4577-a149-0a7a1ac447c6" containerName="authorino" Jun 08 23:06:23.465365 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.465338 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3328e962-5566-4bc1-8158-847e730e4e82" containerName="manager" Jun 08 23:06:23.465365 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.465346 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3328e962-5566-4bc1-8158-847e730e4e82" containerName="manager" Jun 08 23:06:23.465483 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.465430 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3328e962-5566-4bc1-8158-847e730e4e82" containerName="manager" Jun 08 23:06:23.465483 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.465446 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb00cb9e-b8dd-4577-a149-0a7a1ac447c6" containerName="authorino" Jun 08 23:06:23.469693 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.469674 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:23.472727 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.472682 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 23:06:23.472892 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.472762 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zzhmn\"" Jun 08 23:06:23.472892 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.472683 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 23:06:23.476367 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.476339 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84dfc9bfcb-mxrsv"] Jun 08 23:06:23.579256 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.579219 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5b6fc3e-249c-49e7-b791-1fe9fc2af17f-webhook-cert\") pod \"maas-controller-84dfc9bfcb-mxrsv\" (UID: \"c5b6fc3e-249c-49e7-b791-1fe9fc2af17f\") " pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:23.579256 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.579259 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbbxp\" (UniqueName: \"kubernetes.io/projected/c5b6fc3e-249c-49e7-b791-1fe9fc2af17f-kube-api-access-mbbxp\") pod \"maas-controller-84dfc9bfcb-mxrsv\" (UID: \"c5b6fc3e-249c-49e7-b791-1fe9fc2af17f\") " pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:23.680749 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.680713 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5b6fc3e-249c-49e7-b791-1fe9fc2af17f-webhook-cert\") pod \"maas-controller-84dfc9bfcb-mxrsv\" (UID: \"c5b6fc3e-249c-49e7-b791-1fe9fc2af17f\") " pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:23.680908 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.680755 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbbxp\" (UniqueName: \"kubernetes.io/projected/c5b6fc3e-249c-49e7-b791-1fe9fc2af17f-kube-api-access-mbbxp\") pod \"maas-controller-84dfc9bfcb-mxrsv\" (UID: \"c5b6fc3e-249c-49e7-b791-1fe9fc2af17f\") " pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:23.683179 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.683157 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5b6fc3e-249c-49e7-b791-1fe9fc2af17f-webhook-cert\") pod \"maas-controller-84dfc9bfcb-mxrsv\" (UID: \"c5b6fc3e-249c-49e7-b791-1fe9fc2af17f\") " pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:23.688056 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.688035 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbbxp\" (UniqueName: \"kubernetes.io/projected/c5b6fc3e-249c-49e7-b791-1fe9fc2af17f-kube-api-access-mbbxp\") pod \"maas-controller-84dfc9bfcb-mxrsv\" (UID: \"c5b6fc3e-249c-49e7-b791-1fe9fc2af17f\") " pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:23.781924 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.781883 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:23.911867 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:23.911837 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84dfc9bfcb-mxrsv"] Jun 08 23:06:23.913044 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:06:23.913008 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b6fc3e_249c_49e7_b791_1fe9fc2af17f.slice/crio-0d224060059a6390e1f31fffe94e91aab50681568dfb1dbffbb449d5cdefa0bc WatchSource:0}: Error finding container 0d224060059a6390e1f31fffe94e91aab50681568dfb1dbffbb449d5cdefa0bc: Status 404 returned error can't find the container with id 0d224060059a6390e1f31fffe94e91aab50681568dfb1dbffbb449d5cdefa0bc Jun 08 23:06:24.627158 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:24.627069 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3328e962-5566-4bc1-8158-847e730e4e82" path="/var/lib/kubelet/pods/3328e962-5566-4bc1-8158-847e730e4e82/volumes" Jun 08 23:06:24.721087 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:24.721046 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" event={"ID":"c5b6fc3e-249c-49e7-b791-1fe9fc2af17f","Type":"ContainerStarted","Data":"76ee69a820bb3823f724187f94e7b20394d6031e9996b6bdf098368351284d6e"} Jun 08 23:06:24.721287 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:24.721102 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" event={"ID":"c5b6fc3e-249c-49e7-b791-1fe9fc2af17f","Type":"ContainerStarted","Data":"0d224060059a6390e1f31fffe94e91aab50681568dfb1dbffbb449d5cdefa0bc"} Jun 08 23:06:24.721287 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:24.721178 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:24.735609 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:24.735554 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" podStartSLOduration=1.2806104440000001 podStartE2EDuration="1.735537194s" totalImagesPullingTime="454.92675ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:06:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:06:24.734699102 +0000 UTC m=+1106.543310956" watchObservedRunningTime="2026-06-08 23:06:24.735537194 +0000 UTC m=+1106.544149048" Jun 08 23:06:35.733753 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:35.733716 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84dfc9bfcb-mxrsv" Jun 08 23:06:42.557767 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:42.557732 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:06:43.753641 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:06:43.753590 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:03.188655 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:03.188622 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:03.287767 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:03.287729 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:03.510901 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:03.510805 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:03.898455 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:03.898371 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:04.001905 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:04.001864 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:11.730008 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:11.729904 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:17.365881 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:17.365840 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:17.423237 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:17.423204 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:49.473095 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:49.473058 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:49.566707 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:49.566677 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:49.678682 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:49.678645 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:50.106365 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:50.106321 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:50.208177 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:50.208132 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:07:58.679162 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:58.679132 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 23:07:58.683491 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:58.683461 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 23:07:58.683642 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:58.683604 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 23:07:58.688055 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:07:58.688038 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 23:08:13.433568 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:13.433530 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:19.184783 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:19.184751 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:19.288330 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:19.288294 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:19.388467 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:19.388436 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:19.544863 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:19.544770 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:19.647822 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:19.647790 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:29.710693 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:29.710646 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:36.013713 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:36.013677 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:36.132013 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:36.131972 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:36.231886 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:36.231853 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:42.040059 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:42.039954 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:42.096918 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:42.096865 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:50.180989 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:50.180946 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:50.281521 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:50.281479 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:50.382987 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:50.382953 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:08:50.481127 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:08:50.481028 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:06.891298 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:06.891268 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:06.997916 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:06.997856 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:07.108787 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:07.108747 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:12.721158 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:12.721125 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:12.771936 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:12.771905 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:33.042919 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:33.042871 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:33.144890 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:33.144831 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:33.242635 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:33.242577 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:33.731279 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:33.731247 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:33.779597 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:33.779569 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:38.915273 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:38.915240 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:39.016149 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:39.016113 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:39.115162 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:39.115097 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:39.257511 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:39.257476 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:44.771032 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:44.770971 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:09:44.873083 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:09:44.873050 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:11:44.159535 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:11:44.159448 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:11:45.149404 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:11:45.149370 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:12:58.716478 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:12:58.716450 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 23:12:58.721378 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:12:58.721354 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 23:12:58.722946 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:12:58.722930 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 23:12:58.726918 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:12:58.726899 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-138-248.ec2.internal_680c3df85c56e803e7a0410016e00c29/kube-rbac-proxy-crio/0.log" Jun 08 23:14:10.952413 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:10.952364 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59d588cc4c-r82ck_f38d6321-c41d-43f5-8ef5-4f3dac3899a0/authorino/0.log" Jun 08 23:14:15.410224 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:15.410197 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6976cdc864-z5nsf_2d164051-cdfc-45c0-a62d-174d093d5e87/nginx/0.log" Jun 08 23:14:15.758879 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:15.758840 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84dfc9bfcb-mxrsv_c5b6fc3e-249c-49e7-b791-1fe9fc2af17f/manager/0.log" Jun 08 23:14:16.086264 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:16.086192 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56cdd97b9c-f64tz_49ece150-ed54-4dd0-9783-298906f765e6/copy-manifests/0.log" Jun 08 23:14:16.091950 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:16.091927 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56cdd97b9c-f64tz_49ece150-ed54-4dd0-9783-298906f765e6/manager/0.log" Jun 08 23:14:17.074872 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.074843 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r_9f8edcc6-a901-4a51-93bf-bc25c1eef43a/util/0.log" Jun 08 23:14:17.087173 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.087150 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r_9f8edcc6-a901-4a51-93bf-bc25c1eef43a/pull/0.log" Jun 08 23:14:17.093658 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.093631 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r_9f8edcc6-a901-4a51-93bf-bc25c1eef43a/extract/0.log" Jun 08 23:14:17.200993 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.200965 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp_076ea65d-8964-4234-9fa5-8f4801f8b658/util/0.log" Jun 08 23:14:17.207007 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.206981 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp_076ea65d-8964-4234-9fa5-8f4801f8b658/pull/0.log" Jun 08 23:14:17.212869 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.212848 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp_076ea65d-8964-4234-9fa5-8f4801f8b658/extract/0.log" Jun 08 23:14:17.320540 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.320507 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb_836e726b-c5b2-40d9-9673-bee64e44e278/util/0.log" Jun 08 23:14:17.326535 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.326482 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb_836e726b-c5b2-40d9-9673-bee64e44e278/pull/0.log" Jun 08 23:14:17.332615 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.332598 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb_836e726b-c5b2-40d9-9673-bee64e44e278/extract/0.log" Jun 08 23:14:17.446879 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.446849 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l_8686956c-bf4f-41c7-ba79-82abc5b759b6/extract/0.log" Jun 08 23:14:17.453278 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.453255 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l_8686956c-bf4f-41c7-ba79-82abc5b759b6/util/0.log" Jun 08 23:14:17.459797 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.459777 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l_8686956c-bf4f-41c7-ba79-82abc5b759b6/pull/0.log" Jun 08 23:14:17.565993 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.565965 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59d588cc4c-r82ck_f38d6321-c41d-43f5-8ef5-4f3dac3899a0/authorino/0.log" Jun 08 23:14:17.774224 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.774196 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-k4kmq_17596fba-079f-449e-86da-e8b9b5bf5512/manager/0.log" Jun 08 23:14:17.885416 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.885370 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-625wf_04a9326a-0b64-40b8-8217-778d1af2668b/kuadrant-console-plugin/0.log" Jun 08 23:14:17.997719 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:17.997691 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dw466_ffab74d5-d160-4e39-83d5-898a91ea4d35/registry-server/0.log" Jun 08 23:14:18.334355 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:18.334325 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-z5cb4_a1ed08a6-1795-4f77-8204-edc8808fbcbc/manager/0.log" Jun 08 23:14:18.662233 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:18.662155 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bczhx6q_2c8258ce-933b-45be-92fd-803b1bc74503/istio-proxy/0.log" Jun 08 23:14:19.103616 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:19.103584 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zggbq_ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6/istio-proxy/1.log" Jun 08 23:14:19.210466 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:19.210437 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-2xbgw_db1d4c01-2583-4c42-b29a-02575e863904/payload-pre-processing/0.log" Jun 08 23:14:19.757605 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:19.757575 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm_f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed/storage-initializer/0.log" Jun 08 23:14:19.764589 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:19.764563 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-s4lmm_f00e5cbf-2fe1-451e-9b6d-8e567c11e5ed/main/0.log" Jun 08 23:14:19.992564 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:19.992532 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7_b6c2b717-006e-496c-b20f-6380fdd563e1/storage-initializer/0.log" Jun 08 23:14:20.002489 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:20.002464 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-c67c7_b6c2b717-006e-496c-b20f-6380fdd563e1/main/0.log" Jun 08 23:14:20.339591 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:20.339557 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-ksnm9_4ae95d94-353c-4e59-a01b-cf3a4da400e3/storage-initializer/0.log" Jun 08 23:14:20.346123 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:20.346098 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-ksnm9_4ae95d94-353c-4e59-a01b-cf3a4da400e3/main/0.log" Jun 08 23:14:24.679891 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:24.679854 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:14:24.734785 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:24.734756 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:14:27.656965 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:27.656935 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xs9jd_c44e313d-1c49-4766-9a6b-daa3e68b9c2e/global-pull-secret-syncer/0.log" Jun 08 23:14:27.698747 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:27.698722 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" Jun 08 23:14:27.776926 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:27.776901 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-248.ec2.internal_d9b52f40b5c1741b36fc8e18f573c3f1/haproxy/0.log" Jun 08 23:14:32.375711 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.375665 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r_9f8edcc6-a901-4a51-93bf-bc25c1eef43a/extract/0.log" Jun 08 23:14:32.392692 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.392663 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r_9f8edcc6-a901-4a51-93bf-bc25c1eef43a/util/0.log" Jun 08 23:14:32.410023 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.409997 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lmr8r_9f8edcc6-a901-4a51-93bf-bc25c1eef43a/pull/0.log" Jun 08 23:14:32.431271 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.431243 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp_076ea65d-8964-4234-9fa5-8f4801f8b658/extract/0.log" Jun 08 23:14:32.451694 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.451670 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp_076ea65d-8964-4234-9fa5-8f4801f8b658/util/0.log" Jun 08 23:14:32.469928 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.469905 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zgcpp_076ea65d-8964-4234-9fa5-8f4801f8b658/pull/0.log" Jun 08 23:14:32.491677 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.491656 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb_836e726b-c5b2-40d9-9673-bee64e44e278/extract/0.log" Jun 08 23:14:32.508903 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.508883 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb_836e726b-c5b2-40d9-9673-bee64e44e278/util/0.log" Jun 08 23:14:32.524994 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.524968 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jfppb_836e726b-c5b2-40d9-9673-bee64e44e278/pull/0.log" Jun 08 23:14:32.547951 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.547923 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l_8686956c-bf4f-41c7-ba79-82abc5b759b6/extract/0.log" Jun 08 23:14:32.568279 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.568258 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l_8686956c-bf4f-41c7-ba79-82abc5b759b6/util/0.log" Jun 08 23:14:32.584214 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.584192 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wtv8l_8686956c-bf4f-41c7-ba79-82abc5b759b6/pull/0.log" Jun 08 23:14:32.664624 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.664536 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59d588cc4c-r82ck_f38d6321-c41d-43f5-8ef5-4f3dac3899a0/authorino/0.log" Jun 08 23:14:32.734348 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.734314 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-k4kmq_17596fba-079f-449e-86da-e8b9b5bf5512/manager/0.log" Jun 08 23:14:32.762034 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.762008 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-625wf_04a9326a-0b64-40b8-8217-778d1af2668b/kuadrant-console-plugin/0.log" Jun 08 23:14:32.791843 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.791813 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dw466_ffab74d5-d160-4e39-83d5-898a91ea4d35/registry-server/0.log" Jun 08 23:14:32.936100 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:32.936071 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-z5cb4_a1ed08a6-1795-4f77-8204-edc8808fbcbc/manager/0.log" Jun 08 23:14:34.925277 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:34.925250 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-f946b9758-8rw6s_c00c525e-e22c-4d2e-abaa-da2c72c78a69/metrics-server/0.log" Jun 08 23:14:35.085713 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:35.085687 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq5p4_a03609a4-578a-4cca-80da-563aa1fb2aa7/node-exporter/0.log" Jun 08 23:14:35.101529 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:35.101504 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq5p4_a03609a4-578a-4cca-80da-563aa1fb2aa7/kube-rbac-proxy/0.log" Jun 08 23:14:35.116546 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:35.116524 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq5p4_a03609a4-578a-4cca-80da-563aa1fb2aa7/init-textfile/0.log" Jun 08 23:14:35.443489 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:35.443454 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54845cbdfc-wg854_3ca7e03a-3d4d-4084-acd7-002b7adc5d04/telemeter-client/0.log" Jun 08 23:14:35.460818 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:35.460794 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54845cbdfc-wg854_3ca7e03a-3d4d-4084-acd7-002b7adc5d04/reload/0.log" Jun 08 23:14:35.482306 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:35.482279 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54845cbdfc-wg854_3ca7e03a-3d4d-4084-acd7-002b7adc5d04/kube-rbac-proxy/0.log" Jun 08 23:14:39.645618 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.645414 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p"] Jun 08 23:14:39.649282 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.649256 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p"] Jun 08 23:14:39.649472 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.649364 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.651429 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.651404 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4hdn8\"/\"openshift-service-ca.crt\"" Jun 08 23:14:39.651565 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.651443 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4hdn8\"/\"kube-root-ca.crt\"" Jun 08 23:14:39.651917 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.651899 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4hdn8\"/\"default-dockercfg-bfkrn\"" Jun 08 23:14:39.711897 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.711860 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-podres\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.711897 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.711901 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-sys\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.712122 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.711972 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-proc\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.712122 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.711999 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjlgg\" (UniqueName: \"kubernetes.io/projected/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-kube-api-access-fjlgg\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.712122 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.712042 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-lib-modules\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.813160 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.813121 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-sys\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.813326 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.813188 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-proc\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.813326 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.813226 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjlgg\" (UniqueName: \"kubernetes.io/projected/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-kube-api-access-fjlgg\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.813326 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.813247 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-sys\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.813326 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.813257 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-lib-modules\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.813326 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.813311 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-proc\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.813548 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.813365 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-lib-modules\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.813548 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.813379 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-podres\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.813548 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.813484 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-podres\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.820386 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.820366 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjlgg\" (UniqueName: \"kubernetes.io/projected/5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8-kube-api-access-fjlgg\") pod \"perf-node-gather-daemonset-dd29p\" (UID: \"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:39.961868 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:39.961833 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:40.115348 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:40.115190 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p"] Jun 08 23:14:40.117638 ip-10-0-138-248 kubenswrapper[2567]: W0608 23:14:40.117609 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5313dd7b_2d4c_43c4_abdb_e2fb785c0ab8.slice/crio-c3c6bfbbc1eb80bedd1fa2f0844007a7bc981457ed36cdcdaf36fd1abb1b8003 WatchSource:0}: Error finding container c3c6bfbbc1eb80bedd1fa2f0844007a7bc981457ed36cdcdaf36fd1abb1b8003: Status 404 returned error can't find the container with id c3c6bfbbc1eb80bedd1fa2f0844007a7bc981457ed36cdcdaf36fd1abb1b8003 Jun 08 23:14:40.376404 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:40.376303 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9dmt5_1934d5fb-ba1e-48ec-86ee-ae2e12707285/dns/0.log" Jun 08 23:14:40.391444 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:40.391418 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9dmt5_1934d5fb-ba1e-48ec-86ee-ae2e12707285/kube-rbac-proxy/0.log" Jun 08 23:14:40.448217 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:40.448181 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gpjcx_f0ab127e-d27e-4006-93f8-bb2ba3ce2c6e/dns-node-resolver/0.log" Jun 08 23:14:40.460906 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:40.460875 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" event={"ID":"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8","Type":"ContainerStarted","Data":"c9338a7afb314efcd4ce1cd95cf39232861dd15213caaf9cba8754020ce1057d"} Jun 08 23:14:40.461049 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:40.460926 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" event={"ID":"5313dd7b-2d4c-43c4-abdb-e2fb785c0ab8","Type":"ContainerStarted","Data":"c3c6bfbbc1eb80bedd1fa2f0844007a7bc981457ed36cdcdaf36fd1abb1b8003"} Jun 08 23:14:40.461049 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:40.460970 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:40.474519 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:40.474472 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" podStartSLOduration=1.474457725 podStartE2EDuration="1.474457725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 23:14:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:14:40.473589566 +0000 UTC m=+1602.282201419" watchObservedRunningTime="2026-06-08 23:14:40.474457725 +0000 UTC m=+1602.283069578" Jun 08 23:14:41.048564 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:41.048539 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c45cn_b8c10f1b-1d19-4d08-b676-8434b760c799/node-ca/0.log" Jun 08 23:14:41.950959 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:41.950932 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bczhx6q_2c8258ce-933b-45be-92fd-803b1bc74503/istio-proxy/0.log" Jun 08 23:14:42.090773 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:42.090745 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zggbq_ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6/istio-proxy/0.log" Jun 08 23:14:42.107577 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:42.107552 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zggbq_ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6/istio-proxy/1.log" Jun 08 23:14:42.139003 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:42.138975 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-2xbgw_db1d4c01-2583-4c42-b29a-02575e863904/payload-pre-processing/0.log" Jun 08 23:14:42.786237 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:42.786212 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z79nn_3fa35cc0-fc3e-467e-b9a8-16da413b82ed/serve-healthcheck-canary/0.log" Jun 08 23:14:43.362305 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:43.362269 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sv6sf_cb4c6bc3-bb3f-44d6-8af1-78866aa10258/kube-rbac-proxy/0.log" Jun 08 23:14:43.377444 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:43.377419 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sv6sf_cb4c6bc3-bb3f-44d6-8af1-78866aa10258/exporter/0.log" Jun 08 23:14:43.393794 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:43.393771 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sv6sf_cb4c6bc3-bb3f-44d6-8af1-78866aa10258/extractor/0.log" Jun 08 23:14:45.559995 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:45.559962 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6976cdc864-z5nsf_2d164051-cdfc-45c0-a62d-174d093d5e87/nginx/0.log" Jun 08 23:14:45.690317 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:45.690285 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84dfc9bfcb-mxrsv_c5b6fc3e-249c-49e7-b791-1fe9fc2af17f/manager/0.log" Jun 08 23:14:45.801720 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:45.801696 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56cdd97b9c-f64tz_49ece150-ed54-4dd0-9783-298906f765e6/manager/0.log" Jun 08 23:14:45.817055 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:45.816983 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56cdd97b9c-f64tz_49ece150-ed54-4dd0-9783-298906f765e6/copy-manifests/0.log" Jun 08 23:14:46.476531 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:46.476505 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-dd29p" Jun 08 23:14:47.376047 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:47.376020 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-667ddfcf76-r2gxn_d7e5bd47-1a9d-4727-aae3-3ed1d7fcee44/manager/0.log" Jun 08 23:14:49.180316 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:49.180279 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:14:49.281883 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:49.281848 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-54swc_2946f010-81fb-47ea-a3af-d3605b266d6c/konnectivity-agent/0.log" limit=1024 Jun 08 23:14:53.508259 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:53.508234 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bczhx6q_2c8258ce-933b-45be-92fd-803b1bc74503/istio-proxy/0.log" Jun 08 23:14:53.603538 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:53.603513 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zggbq_ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6/istio-proxy/0.log" Jun 08 23:14:53.620337 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:53.620307 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zggbq_ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6/istio-proxy/1.log" Jun 08 23:14:53.645385 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:53.645342 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-2xbgw_db1d4c01-2583-4c42-b29a-02575e863904/payload-pre-processing/0.log" Jun 08 23:14:54.349760 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.349733 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-djtmb_d30efbda-430f-4750-99ba-2da9000234f3/kube-multus-additional-cni-plugins/0.log" Jun 08 23:14:54.366293 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.366263 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-djtmb_d30efbda-430f-4750-99ba-2da9000234f3/egress-router-binary-copy/0.log" Jun 08 23:14:54.381540 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.381514 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-djtmb_d30efbda-430f-4750-99ba-2da9000234f3/cni-plugins/0.log" Jun 08 23:14:54.408277 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.408252 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-djtmb_d30efbda-430f-4750-99ba-2da9000234f3/bond-cni-plugin/0.log" Jun 08 23:14:54.424613 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.424591 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-djtmb_d30efbda-430f-4750-99ba-2da9000234f3/routeoverride-cni/0.log" Jun 08 23:14:54.441931 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.441898 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-djtmb_d30efbda-430f-4750-99ba-2da9000234f3/whereabouts-cni-bincopy/0.log" Jun 08 23:14:54.455559 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.455537 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-djtmb_d30efbda-430f-4750-99ba-2da9000234f3/whereabouts-cni/0.log" Jun 08 23:14:54.626267 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.626188 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xhk8k_260a4693-1d70-4eea-ac68-9bf0ed2c87d9/kube-multus/0.log" Jun 08 23:14:54.671114 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.671087 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bczhx6q_2c8258ce-933b-45be-92fd-803b1bc74503/istio-proxy/0.log" Jun 08 23:14:54.716341 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.716312 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rds9j_fbb0f393-3467-4695-8c0f-d31e1f735f9c/network-metrics-daemon/0.log" Jun 08 23:14:54.731535 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.731513 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rds9j_fbb0f393-3467-4695-8c0f-d31e1f735f9c/kube-rbac-proxy/0.log" Jun 08 23:14:54.795271 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.795242 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zggbq_ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6/istio-proxy/0.log" Jun 08 23:14:54.813033 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.813005 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zggbq_ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6/istio-proxy/1.log" Jun 08 23:14:54.841723 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:54.841695 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-2xbgw_db1d4c01-2583-4c42-b29a-02575e863904/payload-pre-processing/0.log" Jun 08 23:14:55.676885 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:55.676854 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-controller/0.log" Jun 08 23:14:55.689710 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:55.689683 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/0.log" Jun 08 23:14:55.689845 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:55.689760 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovn-acl-logging/1.log" Jun 08 23:14:55.704288 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:55.704265 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/kube-rbac-proxy-node/0.log" Jun 08 23:14:55.721008 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:55.720987 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/kube-rbac-proxy-ovn-metrics/0.log" Jun 08 23:14:55.735292 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:55.735273 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/northd/0.log" Jun 08 23:14:55.750454 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:55.750435 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/nbdb/0.log" Jun 08 23:14:55.766037 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:55.766019 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/sbdb/0.log" Jun 08 23:14:55.804369 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:55.804339 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9hb5j_1b1f88ae-6de8-4d13-9f67-aa38624ff076/ovnkube-controller/0.log" Jun 08 23:14:56.435236 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:56.435198 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bczhx6q_2c8258ce-933b-45be-92fd-803b1bc74503/istio-proxy/0.log" Jun 08 23:14:56.544843 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:56.544793 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zggbq_ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6/istio-proxy/0.log" Jun 08 23:14:56.563144 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:56.563116 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-zggbq_ab03d9ec-73be-48e6-9fa8-ae7fd9c97ad6/istio-proxy/1.log" Jun 08 23:14:56.588983 ip-10-0-138-248 kubenswrapper[2567]: I0608 23:14:56.588959 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-2xbgw_db1d4c01-2583-4c42-b29a-02575e863904/payload-pre-processing/0.log"