Jun 06 22:02:52.872105 ip-10-0-139-115 systemd[1]: Starting Kubernetes Kubelet... Jun 06 22:02:53.328062 ip-10-0-139-115 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 06 22:02:53.328062 ip-10-0-139-115 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 06 22:02:53.328062 ip-10-0-139-115 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 06 22:02:53.328062 ip-10-0-139-115 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 06 22:02:53.328062 ip-10-0-139-115 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 06 22:02:53.329429 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.328116 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 06 22:02:53.333398 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333379 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:02:53.333398 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333398 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333401 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333405 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333408 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333411 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333415 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333418 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333421 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333423 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333426 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333430 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333432 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333435 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333438 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333441 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333444 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333447 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333449 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333452 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333455 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:02:53.333473 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333458 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333461 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333463 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333466 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333468 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333471 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333473 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333476 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333479 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333482 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333485 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333487 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333490 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333493 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333495 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333498 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333500 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333503 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333505 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:02:53.333974 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333508 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333510 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333513 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333515 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333518 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333520 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333523 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333525 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333528 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333531 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333534 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333537 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333539 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333542 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333546 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333551 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333554 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333557 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333561 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333564 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:02:53.334431 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333567 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333570 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333573 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333576 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333579 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333581 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333584 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333586 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333590 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333595 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333598 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333600 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333603 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333606 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333608 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333611 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333613 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333616 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333619 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333622 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:02:53.334933 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333624 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333627 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333629 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333633 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333635 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333638 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333641 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333645 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333648 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333651 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333653 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333656 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333659 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333661 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333664 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333667 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333669 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333672 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.333674 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:02:53.335423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334393 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334401 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334404 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334407 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334410 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334412 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334415 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334420 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334423 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334426 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334428 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334431 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334433 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334436 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334439 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334442 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334445 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334448 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334450 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:02:53.335944 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334453 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334456 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334459 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334462 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334465 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334467 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334470 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334473 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334476 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334478 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334481 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334483 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334486 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334488 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334491 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334493 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334496 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334498 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334501 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334504 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:02:53.336423 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334506 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334509 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334511 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334513 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334516 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334519 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334522 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334524 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334527 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334529 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334531 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334535 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334537 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334540 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334543 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334545 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334548 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334551 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334553 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334555 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:02:53.336930 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334558 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334560 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334563 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334566 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334568 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334570 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334574 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334576 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334579 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334581 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334584 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334586 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334589 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334591 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334594 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334597 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334599 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334602 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334605 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:02:53.337434 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334609 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334613 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334617 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334620 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334623 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334627 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334631 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334633 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334636 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334639 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334641 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334644 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334646 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334649 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334652 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334654 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334657 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334660 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334663 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:02:53.338009 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334666 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.334669 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335057 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335062 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335065 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335068 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335071 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335074 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335076 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335079 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335083 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335086 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335088 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335093 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335097 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335100 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335103 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335106 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335109 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:02:53.338479 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335112 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335115 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335118 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335122 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335124 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335127 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335130 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335133 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335136 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335138 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335141 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335143 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335146 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335149 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335151 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335154 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335156 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335159 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335162 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335164 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:02:53.338973 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335167 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335169 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335172 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335176 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335180 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335182 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335185 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335188 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335191 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335193 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335196 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335198 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335201 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335204 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335207 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335209 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335212 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335215 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335217 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335220 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:02:53.339476 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335222 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335226 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335229 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335232 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335235 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335238 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335241 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335243 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335246 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335248 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335251 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335253 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335256 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335258 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335261 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335263 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335266 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335268 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335270 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335273 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:02:53.339994 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335278 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335280 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335283 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335285 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335290 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335293 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335295 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335298 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335301 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335303 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335306 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335308 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335310 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335313 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335315 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335351 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335355 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335358 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335361 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335364 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:02:53.340480 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335366 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.335369 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335439 2580 flags.go:64] FLAG: --address="0.0.0.0" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335448 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335457 2580 flags.go:64] FLAG: --anonymous-auth="true" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335463 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335469 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335474 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335481 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335485 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335488 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335492 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335495 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335499 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335502 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335505 2580 flags.go:64] FLAG: --cgroup-root="" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335509 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335512 2580 flags.go:64] FLAG: --client-ca-file="" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335515 2580 flags.go:64] FLAG: --cloud-provider="external" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335518 2580 flags.go:64] FLAG: --cluster-dns="[]" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335522 2580 flags.go:64] FLAG: --cluster-domain="" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335525 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335528 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 06 22:02:53.340980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335531 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335535 2580 flags.go:64] FLAG: --container-log-max-files="5" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335538 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335541 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335544 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335548 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335551 2580 flags.go:64] FLAG: --contention-profiling="false" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335554 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335557 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335560 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335563 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335567 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335570 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335573 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335576 2580 flags.go:64] FLAG: --enable-load-reader="false" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335579 2580 flags.go:64] FLAG: --enable-server="true" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335582 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335586 2580 flags.go:64] FLAG: --event-burst="100" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335589 2580 flags.go:64] FLAG: --event-qps="50" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335592 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335595 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335598 2580 flags.go:64] FLAG: --eviction-hard="" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335608 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335611 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335614 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 06 22:02:53.341539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335617 2580 flags.go:64] FLAG: --eviction-soft="" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335620 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335623 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335627 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335630 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335633 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335636 2580 flags.go:64] FLAG: --fail-swap-on="true" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335638 2580 flags.go:64] FLAG: --feature-gates="" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335642 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335645 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335648 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335651 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335654 2580 flags.go:64] FLAG: --healthz-port="10248" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335657 2580 flags.go:64] FLAG: --help="false" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335660 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335663 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335666 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335669 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335672 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335676 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335679 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335681 2580 flags.go:64] FLAG: --image-service-endpoint="" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335684 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 06 22:02:53.342174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335687 2580 flags.go:64] FLAG: --kube-api-burst="100" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335694 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335697 2580 flags.go:64] FLAG: --kube-api-qps="50" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335700 2580 flags.go:64] FLAG: --kube-reserved="" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335703 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335706 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335709 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335712 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335715 2580 flags.go:64] FLAG: --lock-file="" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335718 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335721 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335724 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335729 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335733 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335736 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335738 2580 flags.go:64] FLAG: --logging-format="text" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335741 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335745 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335747 2580 flags.go:64] FLAG: --manifest-url="" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335750 2580 flags.go:64] FLAG: --manifest-url-header="" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335754 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335757 2580 flags.go:64] FLAG: --max-open-files="1000000" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335761 2580 flags.go:64] FLAG: --max-pods="110" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335764 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335767 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 06 22:02:53.342833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335770 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335773 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335776 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335779 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335782 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335789 2580 flags.go:64] FLAG: --node-status-max-images="50" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335792 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335795 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335814 2580 flags.go:64] FLAG: --pod-cidr="" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335818 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335823 2580 flags.go:64] FLAG: --pod-manifest-path="" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335826 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335830 2580 flags.go:64] FLAG: --pods-per-core="0" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335833 2580 flags.go:64] FLAG: --port="10250" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335836 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335838 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09945bb25eb0fe590" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335842 2580 flags.go:64] FLAG: --qos-reserved="" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335845 2580 flags.go:64] FLAG: --read-only-port="10255" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335848 2580 flags.go:64] FLAG: --register-node="true" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335851 2580 flags.go:64] FLAG: --register-with-taints="" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335855 2580 flags.go:64] FLAG: --registry-burst="10" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335858 2580 flags.go:64] FLAG: --registry-qps="5" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335861 2580 flags.go:64] FLAG: --reserved-cpus="" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335864 2580 flags.go:64] FLAG: --reserved-memory="" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335867 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 06 22:02:53.343461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335870 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335873 2580 flags.go:64] FLAG: --rotate-certificates="false" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335876 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335879 2580 flags.go:64] FLAG: --runonce="false" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335881 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335884 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335887 2580 flags.go:64] FLAG: --seccomp-default="false" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335890 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335893 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335896 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335899 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335902 2580 flags.go:64] FLAG: --storage-driver-password="root" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335905 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335908 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335910 2580 flags.go:64] FLAG: --storage-driver-user="root" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335915 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335919 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335921 2580 flags.go:64] FLAG: --system-cgroups="" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335925 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335930 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335933 2580 flags.go:64] FLAG: --tls-cert-file="" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335936 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335939 2580 flags.go:64] FLAG: --tls-min-version="" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335942 2580 flags.go:64] FLAG: --tls-private-key-file="" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335945 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jun 06 22:02:53.344125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335948 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335952 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335955 2580 flags.go:64] FLAG: --v="2" Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335959 2580 flags.go:64] FLAG: --version="false" Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335964 2580 flags.go:64] FLAG: --vmodule="" Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335967 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.335971 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336110 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336115 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336118 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336121 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336124 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336127 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336130 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336133 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336136 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336139 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336142 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336145 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336148 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336150 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:02:53.344743 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336153 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336157 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336161 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336163 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336166 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336169 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336171 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336174 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336177 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336180 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336182 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336185 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336188 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336190 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336193 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336195 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336199 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336201 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336204 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336206 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:02:53.345318 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336209 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336211 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336214 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336218 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336222 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336225 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336227 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336230 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336233 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336235 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336238 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336240 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336244 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336249 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336252 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336255 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336258 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336260 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:02:53.345824 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336263 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336265 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336268 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336270 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336273 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336275 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336278 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336280 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336283 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336285 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336288 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336291 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336293 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336296 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336298 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336301 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336303 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336305 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336308 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336310 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:02:53.346290 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336314 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336317 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336319 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336322 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336325 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336328 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336330 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336338 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336341 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336343 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336345 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336348 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336351 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336353 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336356 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336358 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336360 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336363 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336365 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336368 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:02:53.346785 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336371 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336374 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336376 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336379 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336382 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336384 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.336387 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.337072 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.344346 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.344365 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.344379 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.344388 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344448 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344453 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:02:53.347275 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344456 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344459 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344462 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344466 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344469 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344472 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344475 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344477 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344480 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344483 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344485 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344488 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344490 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344493 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344496 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344498 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344501 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344504 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344506 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344509 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:02:53.347621 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344514 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344517 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344521 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344524 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344528 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344530 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344533 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344543 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344546 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344549 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344552 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344555 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344557 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344560 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344563 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344565 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344568 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344570 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344573 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344576 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:02:53.348110 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344578 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344581 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344584 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344586 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344589 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344591 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344594 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344596 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344599 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344602 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344604 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344607 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344609 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344612 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344614 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344616 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344619 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344621 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344624 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344628 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:02:53.348615 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344637 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344640 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344643 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344645 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344648 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344650 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344653 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344657 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344661 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344663 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344666 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344669 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344672 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344674 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344677 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344679 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344682 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344684 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344687 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:02:53.349116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344690 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344692 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344695 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344697 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344700 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344702 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344705 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344707 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344710 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344712 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344715 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344717 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344721 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344724 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344732 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344735 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344737 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:02:53.349566 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344740 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.344744 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344911 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344918 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344921 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344923 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344926 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344929 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344932 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344934 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344937 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344939 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344942 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344944 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 06 22:02:53.350044 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344947 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344949 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344953 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344957 2580 feature_gate.go:328] unrecognized feature gate: Example Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344960 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344964 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344967 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344970 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344972 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344975 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344978 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344981 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344983 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344987 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344989 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.344992 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345003 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345006 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345009 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 06 22:02:53.350395 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345012 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345015 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345017 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345020 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345023 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345026 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345028 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345031 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345034 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345036 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345039 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345041 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345044 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345046 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345049 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345052 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345054 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345057 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345059 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345062 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 06 22:02:53.351004 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345064 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345068 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345072 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345075 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345077 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345080 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345083 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345085 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345088 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345091 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345100 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345103 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345105 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345107 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345110 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345113 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345115 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345118 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345120 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 06 22:02:53.351498 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345122 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345125 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345127 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345130 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345133 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345135 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345138 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345140 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345143 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345145 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345148 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345150 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345153 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345155 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345158 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345160 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345163 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345165 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345168 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345171 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 06 22:02:53.351963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345174 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345176 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345179 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345181 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345189 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345192 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345195 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345197 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:53.345200 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.345205 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.345352 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.349484 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.351202 2580 server.go:1019] "Starting client certificate rotation" Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.351313 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 06 22:02:53.352442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.351989 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 06 22:02:53.356657 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.356638 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 06 22:02:53.357762 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.357745 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 06 22:02:53.368449 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.368434 2580 log.go:25] "Validated CRI v1 runtime API" Jun 06 22:02:53.372385 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.372363 2580 log.go:25] "Validated CRI v1 image API" Jun 06 22:02:53.373233 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.373217 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 06 22:02:53.376691 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.376670 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9a7c609a-8257-43cc-8cb1-bbaec4ca144a:/dev/nvme0n1p3 c4a526ef-6ac9-46bf-bfd9-eb1cf79ba4ad:/dev/nvme0n1p4] Jun 06 22:02:53.376766 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.376691 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 06 22:02:53.379371 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.379347 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 06 22:02:53.382300 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.382179 2580 manager.go:216] Machine: {Timestamp:2026-06-06 22:02:53.380691372 +0000 UTC m=+0.379670846 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099318 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28a2d1fa257feaad93edec9b42aae0 SystemUUID:ec28a2d1-fa25-7fea-ad93-edec9b42aae0 BootID:54dfd15a-d10b-43cb-b654-49e019a13df3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:77:5c:9e:5f:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:77:5c:9e:5f:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:1f:f2:5c:d9:9d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 06 22:02:53.382300 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.382293 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 06 22:02:53.382459 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.382392 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 06 22:02:53.385295 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.385268 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 06 22:02:53.385452 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.385297 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-115.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 06 22:02:53.386236 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.386223 2580 topology_manager.go:138] "Creating topology manager with none policy" Jun 06 22:02:53.386290 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.386240 2580 container_manager_linux.go:309] "Creating device plugin manager" Jun 06 22:02:53.386290 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.386252 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 06 22:02:53.386290 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.386273 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 06 22:02:53.387652 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.387640 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 06 22:02:53.392073 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.392060 2580 state_mem.go:36] "Initialized new in-memory state store" Jun 06 22:02:53.392202 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.392191 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 06 22:02:53.392256 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.392229 2580 kubelet.go:486] "Attempting to sync node with API server" Jun 06 22:02:53.392256 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.392244 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 06 22:02:53.392355 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.392263 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 06 22:02:53.392355 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.392318 2580 kubelet.go:398] "Adding apiserver pod source" Jun 06 22:02:53.392355 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.392331 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 06 22:02:53.392714 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.392700 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 06 22:02:53.392775 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.392722 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 06 22:02:53.395150 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.395135 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 06 22:02:53.397001 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.396988 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 06 22:02:53.397039 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.397018 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 06 22:02:53.398321 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398310 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 06 22:02:53.398362 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398326 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 06 22:02:53.398362 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398343 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 06 22:02:53.398362 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398349 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 06 22:02:53.398362 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398359 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 06 22:02:53.398464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398364 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 06 22:02:53.398464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398373 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 06 22:02:53.398464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398378 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 06 22:02:53.398464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398384 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 06 22:02:53.398464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398395 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 06 22:02:53.398464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398409 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 06 22:02:53.398464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.398414 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 06 22:02:53.399299 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.399288 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 06 22:02:53.399299 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.399298 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 06 22:02:53.402313 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.402296 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-drqvd" Jun 06 22:02:53.402516 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.402502 2580 server.go:1268] "Started kubelet" Jun 06 22:02:53.403233 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.403166 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 06 22:02:53.403317 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.403283 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 06 22:02:53.403975 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.403932 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 06 22:02:53.404106 ip-10-0-139-115 systemd[1]: Started Kubernetes Kubelet. Jun 06 22:02:53.405689 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.405669 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 06 22:02:53.406096 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.406073 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-115.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 06 22:02:53.406167 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.406130 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 06 22:02:53.407728 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.407710 2580 server.go:310] "Adding debug handlers to kubelet server" Jun 06 22:02:53.411445 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.411423 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-drqvd" Jun 06 22:02:53.412001 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.411980 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 06 22:02:53.413011 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.412990 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 06 22:02:53.413987 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.412838 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-115.ec2.internal.18b69d0b099385d8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-115.ec2.internal,UID:ip-10-0-139-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-115.ec2.internal,},FirstTimestamp:2026-06-06 22:02:53.402473944 +0000 UTC m=+0.401453418,LastTimestamp:2026-06-06 22:02:53.402473944 +0000 UTC m=+0.401453418,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-115.ec2.internal,}" Jun 06 22:02:53.415202 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.415184 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 06 22:02:53.415202 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.415194 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 06 22:02:53.415703 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.415327 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:53.416115 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416100 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 06 22:02:53.416115 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416116 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 06 22:02:53.416247 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416124 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 06 22:02:53.416247 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416233 2580 reconstruct.go:97] "Volume reconstruction finished" Jun 06 22:02:53.416247 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416241 2580 reconciler.go:29] "Reconciler: start to sync state" Jun 06 22:02:53.416381 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416322 2580 factory.go:55] Registering systemd factory Jun 06 22:02:53.416381 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416342 2580 factory.go:223] Registration of the systemd container factory successfully Jun 06 22:02:53.416558 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416531 2580 factory.go:153] Registering CRI-O factory Jun 06 22:02:53.416558 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416555 2580 factory.go:223] Registration of the crio container factory successfully Jun 06 22:02:53.416663 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416610 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 06 22:02:53.416663 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416633 2580 factory.go:103] Registering Raw factory Jun 06 22:02:53.416663 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.416649 2580 manager.go:1196] Started watching for new ooms in manager Jun 06 22:02:53.417378 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.417266 2580 manager.go:349] Starting recovery of all containers Jun 06 22:02:53.420789 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.418238 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 06 22:02:53.420789 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.418276 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 06 22:02:53.426227 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.426065 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 06 22:02:53.427825 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.427779 2580 manager.go:354] Recovery completed Jun 06 22:02:53.429588 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.429558 2580 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 06 22:02:53.432397 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.432386 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:02:53.434166 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.434151 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:02:53.434236 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.434180 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:02:53.434236 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.434190 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:02:53.434700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.434688 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 06 22:02:53.434700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.434699 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 06 22:02:53.434780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.434701 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:02:53.434780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.434713 2580 state_mem.go:36] "Initialized new in-memory state store" Jun 06 22:02:53.435392 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.435378 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:02:53.435430 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.435404 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:02:53.435430 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.435413 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:02:53.436712 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.436696 2580 policy_none.go:49] "None policy: Start" Jun 06 22:02:53.436712 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.436713 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 06 22:02:53.436840 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.436724 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 06 22:02:53.438523 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.438511 2580 policy_none.go:47] "Start" Jun 06 22:02:53.474470 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.474453 2580 manager.go:337] "Starting Device Plugin manager" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.474493 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.474506 2580 server.go:87] "Starting device plugin registration server" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.474757 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.474768 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.474866 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.474971 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.474982 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.475402 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.475441 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.482236 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.482267 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.482286 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.482322 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 06 22:02:53.494667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.484404 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:02:53.575249 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.575212 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:02:53.576532 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.576514 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:02:53.576613 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.576545 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:02:53.576613 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.576558 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:02:53.576613 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.576581 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.582662 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.582618 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal"] Jun 06 22:02:53.582713 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.582676 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:02:53.584187 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.584174 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.584264 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.584194 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-115.ec2.internal\": node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:53.584264 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.584176 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:02:53.584339 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.584271 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:02:53.584339 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.584283 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:02:53.586827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.586798 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:02:53.586949 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.586934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.586993 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.586971 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:02:53.588515 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.588490 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:02:53.588615 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.588521 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:02:53.588615 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.588536 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:02:53.588615 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.588492 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:02:53.588615 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.588590 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:02:53.588615 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.588600 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:02:53.590824 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.590795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.590906 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.590840 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 06 22:02:53.591536 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.591522 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientMemory" Jun 06 22:02:53.591605 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.591545 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasNoDiskPressure" Jun 06 22:02:53.591605 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.591555 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeHasSufficientPID" Jun 06 22:02:53.601376 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.601354 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:53.605325 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.605304 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-115.ec2.internal\" not found" node="ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.609460 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.609445 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-115.ec2.internal\" not found" node="ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.617890 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.617874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/809d1195e782cf85c5467f6d560ad47d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal\" (UID: \"809d1195e782cf85c5467f6d560ad47d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.617973 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.617896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/809d1195e782cf85c5467f6d560ad47d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal\" (UID: \"809d1195e782cf85c5467f6d560ad47d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.617973 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.617912 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2f312d8097474ef9a1a11d8e634ff7d8-config\") pod \"kube-apiserver-proxy-ip-10-0-139-115.ec2.internal\" (UID: \"2f312d8097474ef9a1a11d8e634ff7d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.701631 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.701592 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:53.718195 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.718174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/809d1195e782cf85c5467f6d560ad47d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal\" (UID: \"809d1195e782cf85c5467f6d560ad47d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.718310 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.718217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/809d1195e782cf85c5467f6d560ad47d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal\" (UID: \"809d1195e782cf85c5467f6d560ad47d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.718310 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.718234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2f312d8097474ef9a1a11d8e634ff7d8-config\") pod \"kube-apiserver-proxy-ip-10-0-139-115.ec2.internal\" (UID: \"2f312d8097474ef9a1a11d8e634ff7d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.718310 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.718279 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2f312d8097474ef9a1a11d8e634ff7d8-config\") pod \"kube-apiserver-proxy-ip-10-0-139-115.ec2.internal\" (UID: \"2f312d8097474ef9a1a11d8e634ff7d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.718310 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.718285 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/809d1195e782cf85c5467f6d560ad47d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal\" (UID: \"809d1195e782cf85c5467f6d560ad47d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.718310 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.718289 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/809d1195e782cf85c5467f6d560ad47d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal\" (UID: \"809d1195e782cf85c5467f6d560ad47d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.802536 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.802507 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:53.903239 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:53.903214 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:53.911327 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.911307 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" Jun 06 22:02:53.916148 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:53.916065 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal" Jun 06 22:02:54.003348 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:54.003306 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:54.103829 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:54.103785 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:54.204389 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:54.204314 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:54.263570 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.263546 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:02:54.304581 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:54.304559 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:54.350546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.350516 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 06 22:02:54.351328 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.350684 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 06 22:02:54.351328 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.350701 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 06 22:02:54.405038 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:54.404996 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-115.ec2.internal\" not found" Jun 06 22:02:54.416786 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.416762 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:02:54.420165 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.420138 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-05 21:57:53 +0000 UTC" deadline="2027-12-21 17:43:14.580652085 +0000 UTC" Jun 06 22:02:54.420165 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.420163 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13507h40m20.160491953s" Jun 06 22:02:54.433258 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.433240 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:02:54.437445 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:54.437417 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f312d8097474ef9a1a11d8e634ff7d8.slice/crio-c26e5da244157216dd32a73404eaf49e74846cab0ba1393f5920c7382b5f45c4 WatchSource:0}: Error finding container c26e5da244157216dd32a73404eaf49e74846cab0ba1393f5920c7382b5f45c4: Status 404 returned error can't find the container with id c26e5da244157216dd32a73404eaf49e74846cab0ba1393f5920c7382b5f45c4 Jun 06 22:02:54.437759 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:54.437741 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809d1195e782cf85c5467f6d560ad47d.slice/crio-5fc02aad8554e9890248b5fad08205948d3413342f97c945f04d08ecd7366580 WatchSource:0}: Error finding container 5fc02aad8554e9890248b5fad08205948d3413342f97c945f04d08ecd7366580: Status 404 returned error can't find the container with id 5fc02aad8554e9890248b5fad08205948d3413342f97c945f04d08ecd7366580 Jun 06 22:02:54.442005 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.441988 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 06 22:02:54.484859 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.484767 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" event={"ID":"809d1195e782cf85c5467f6d560ad47d","Type":"ContainerStarted","Data":"5fc02aad8554e9890248b5fad08205948d3413342f97c945f04d08ecd7366580"} Jun 06 22:02:54.485781 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.485761 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal" event={"ID":"2f312d8097474ef9a1a11d8e634ff7d8","Type":"ContainerStarted","Data":"c26e5da244157216dd32a73404eaf49e74846cab0ba1393f5920c7382b5f45c4"} Jun 06 22:02:54.516734 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.516711 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" Jun 06 22:02:54.527547 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.527530 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 06 22:02:54.528694 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.528682 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal" Jun 06 22:02:54.534867 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:54.534855 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 06 22:02:55.393171 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.393135 2580 apiserver.go:52] "Watching apiserver" Jun 06 22:02:55.402525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.402502 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 06 22:02:55.402997 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.402966 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-vlk2c","kube-system/global-pull-secret-syncer-m7gsp","kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9dxx6","openshift-dns/node-resolver-bp8js","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal","openshift-multus/multus-srh2f","openshift-network-operator/iptables-alerter-99x6w","openshift-ovn-kubernetes/ovnkube-node-74znx","kube-system/konnectivity-agent-bkvtn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k","openshift-image-registry/node-ca-txkmb","openshift-multus/multus-additional-cni-plugins-hwmq5","openshift-multus/network-metrics-daemon-j7ksf"] Jun 06 22:02:55.405733 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.405713 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.407938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.407918 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.408427 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.408406 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xb4wr\"" Jun 06 22:02:55.408427 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.408424 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 06 22:02:55.408572 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.408422 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 06 22:02:55.408572 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.408471 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 06 22:02:55.408779 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.408765 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 06 22:02:55.410154 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.410136 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:55.410247 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.410211 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 06 22:02:55.410247 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.410215 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:02:55.410360 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.410140 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 06 22:02:55.410590 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.410570 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s9wd7\"" Jun 06 22:02:55.412851 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.412832 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.415241 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.415223 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xk8f8\"" Jun 06 22:02:55.415425 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.415240 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:02:55.415425 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.415258 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 06 22:02:55.415537 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.415520 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 06 22:02:55.415579 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.415560 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.418036 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.418018 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 06 22:02:55.418125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.418089 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-txkvz\"" Jun 06 22:02:55.418182 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.418121 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 06 22:02:55.418544 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.418524 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.421158 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.421138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.421512 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.421489 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jmsq8\"" Jun 06 22:02:55.421591 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.421537 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:02:55.421652 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.421596 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 06 22:02:55.423179 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.423162 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 06 22:02:55.423451 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.423434 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:02:55.423559 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.423532 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:02:55.423636 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.423613 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 06 22:02:55.423706 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.423691 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 06 22:02:55.423778 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.423716 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 06 22:02:55.424467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.424358 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 06 22:02:55.424467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.424379 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 06 22:02:55.424617 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.424490 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fqxbh\"" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ptjk\" (UniqueName: \"kubernetes.io/projected/22adddad-6c0c-4608-9c53-45b0644d677a-kube-api-access-5ptjk\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425267 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-modprobe-d\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-var-lib-kubelet\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425347 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425438 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9j6d\" (UniqueName: \"kubernetes.io/projected/5f4eec8c-1779-4a2f-b0e0-a028da53560b-kube-api-access-v9j6d\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425471 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-sys\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425523 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-lib-modules\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-daemon-config\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-system-cni-dir\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-cnibin\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425699 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-host\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425733 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-tmp\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425760 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6c24d13-14a9-4f94-8112-0ff9d1f62946-hosts-file\") pod \"node-resolver-bp8js\" (UID: \"c6c24d13-14a9-4f94-8112-0ff9d1f62946\") " pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425787 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425817 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqbcc\" (UniqueName: \"kubernetes.io/projected/f5b26be1-08e7-4b34-a136-92a23404cb19-kube-api-access-vqbcc\") pod \"iptables-alerter-99x6w\" (UID: \"f5b26be1-08e7-4b34-a136-92a23404cb19\") " pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425850 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22adddad-6c0c-4608-9c53-45b0644d677a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.426191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-sysctl-conf\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.425965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-tuned\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426024 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-cni-binary-copy\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-run-netns\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-var-lib-cni-multus\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-var-lib-kubelet\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426441 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-run-multus-certs\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj8pd\" (UniqueName: \"kubernetes.io/projected/c6c24d13-14a9-4f94-8112-0ff9d1f62946-kube-api-access-qj8pd\") pod \"node-resolver-bp8js\" (UID: \"c6c24d13-14a9-4f94-8112-0ff9d1f62946\") " pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426609 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5b26be1-08e7-4b34-a136-92a23404cb19-host-slash\") pod \"iptables-alerter-99x6w\" (UID: \"f5b26be1-08e7-4b34-a136-92a23404cb19\") " pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426686 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-cni-dir\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nfw2\" (UniqueName: \"kubernetes.io/projected/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-kube-api-access-7nfw2\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.426911 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-os-release\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.427060 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427030 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-sysconfig\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427097 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-kubernetes\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427133 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-sysctl-d\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427151 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-run\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-var-lib-cni-bin\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-etc-kubernetes\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427294 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22adddad-6c0c-4608-9c53-45b0644d677a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-cnibin\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-os-release\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427383 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7n5b\" (UniqueName: \"kubernetes.io/projected/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-kube-api-access-f7n5b\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427413 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f5b26be1-08e7-4b34-a136-92a23404cb19-iptables-alerter-script\") pod \"iptables-alerter-99x6w\" (UID: \"f5b26be1-08e7-4b34-a136-92a23404cb19\") " pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427445 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-system-cni-dir\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-socket-dir-parent\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427501 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-run-k8s-cni-cncf-io\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-systemd\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.427684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6c24d13-14a9-4f94-8112-0ff9d1f62946-tmp-dir\") pod \"node-resolver-bp8js\" (UID: \"c6c24d13-14a9-4f94-8112-0ff9d1f62946\") " pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.428478 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-hostroot\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.428478 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427773 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:55.428478 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-conf-dir\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.428478 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.427912 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:02:55.428478 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.427914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22adddad-6c0c-4608-9c53-45b0644d677a-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.429479 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.429460 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:02:55.432063 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.432044 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.432357 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.432337 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 06 22:02:55.432637 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.432621 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dp5xn\"" Jun 06 22:02:55.432637 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.432640 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 06 22:02:55.434364 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.434346 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 06 22:02:55.434364 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.434368 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.434546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.434523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5nr5x\"" Jun 06 22:02:55.434608 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.434537 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 06 22:02:55.435003 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.434951 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 06 22:02:55.436603 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.436581 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 06 22:02:55.436694 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.436632 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xcg98\"" Jun 06 22:02:55.436947 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.436931 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 06 22:02:55.437013 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.437002 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 06 22:02:55.446268 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.446247 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vbqj5" Jun 06 22:02:55.454892 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.454874 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vbqj5" Jun 06 22:02:55.517571 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.517502 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 06 22:02:55.528414 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528378 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-registration-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.528414 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-sysctl-conf\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.528626 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-tuned\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.528626 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528571 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-sysctl-conf\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.528626 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528620 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-cni-binary-copy\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.528824 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528698 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 06 22:02:55.528824 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-run-netns\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.528824 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-var-lib-cni-multus\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.528978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-run-multus-certs\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.528978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528859 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nfw2\" (UniqueName: \"kubernetes.io/projected/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-kube-api-access-7nfw2\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.528978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5b26be1-08e7-4b34-a136-92a23404cb19-host-slash\") pod \"iptables-alerter-99x6w\" (UID: \"f5b26be1-08e7-4b34-a136-92a23404cb19\") " pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.528978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528884 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-run-netns\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.528978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528897 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-var-lib-cni-multus\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.528978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-os-release\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.528978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.528974 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-run-multus-certs\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529001 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0-agent-certs\") pod \"konnectivity-agent-bkvtn\" (UID: \"7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0\") " pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529017 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5b26be1-08e7-4b34-a136-92a23404cb19-host-slash\") pod \"iptables-alerter-99x6w\" (UID: \"f5b26be1-08e7-4b34-a136-92a23404cb19\") " pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-ovn-node-metrics-cert\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529090 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-os-release\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-socket-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529178 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9862602e-db24-4b4a-ab24-a6b16129404e-host\") pod \"node-ca-txkmb\" (UID: \"9862602e-db24-4b4a-ab24-a6b16129404e\") " pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529180 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-cni-binary-copy\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-kubernetes\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529271 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-kubernetes\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.529297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-sysctl-d\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-run\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-etc-kubernetes\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529342 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22adddad-6c0c-4608-9c53-45b0644d677a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-kubelet\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cj6t\" (UniqueName: \"kubernetes.io/projected/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-kube-api-access-6cj6t\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529402 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-etc-kubernetes\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529433 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-sysctl-d\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-cnibin\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7n5b\" (UniqueName: \"kubernetes.io/projected/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-kube-api-access-f7n5b\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529488 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-run\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529489 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-cnibin\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529499 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f5b26be1-08e7-4b34-a136-92a23404cb19-iptables-alerter-script\") pod \"iptables-alerter-99x6w\" (UID: \"f5b26be1-08e7-4b34-a136-92a23404cb19\") " pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529536 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-system-cni-dir\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529566 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-run-netns\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529591 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-sys-fs\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.529709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxqq4\" (UniqueName: \"kubernetes.io/projected/987b4ea0-fe2b-48af-8c3e-608f7109cb57-kube-api-access-hxqq4\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-hostroot\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-conf-dir\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22adddad-6c0c-4608-9c53-45b0644d677a-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529720 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-run-systemd\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529727 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-system-cni-dir\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529743 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-env-overrides\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529774 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-conf-dir\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9862602e-db24-4b4a-ab24-a6b16129404e-serviceca\") pod \"node-ca-txkmb\" (UID: \"9862602e-db24-4b4a-ab24-a6b16129404e\") " pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-modprobe-d\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-var-lib-kubelet\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-ovnkube-config\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530001 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpj6z\" (UniqueName: \"kubernetes.io/projected/9862602e-db24-4b4a-ab24-a6b16129404e-kube-api-access-mpj6z\") pod \"node-ca-txkmb\" (UID: \"9862602e-db24-4b4a-ab24-a6b16129404e\") " pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530017 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-hostroot\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-lib-modules\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.530546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.529976 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22adddad-6c0c-4608-9c53-45b0644d677a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530035 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-modprobe-d\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530059 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f5b26be1-08e7-4b34-a136-92a23404cb19-iptables-alerter-script\") pod \"iptables-alerter-99x6w\" (UID: \"f5b26be1-08e7-4b34-a136-92a23404cb19\") " pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530077 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-system-cni-dir\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530098 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-var-lib-kubelet\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-systemd-units\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.530130 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530149 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-etc-openvswitch\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530161 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-lib-modules\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530177 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-system-cni-dir\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.530201 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs podName:5f4eec8c-1779-4a2f-b0e0-a028da53560b nodeName:}" failed. No retries permitted until 2026-06-06 22:02:56.030182898 +0000 UTC m=+3.029162381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs") pod "network-metrics-daemon-j7ksf" (UID: "5f4eec8c-1779-4a2f-b0e0-a028da53560b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-host\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-tmp\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6c24d13-14a9-4f94-8112-0ff9d1f62946-hosts-file\") pod \"node-resolver-bp8js\" (UID: \"c6c24d13-14a9-4f94-8112-0ff9d1f62946\") " pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530276 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-host\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-var-lib-kubelet\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530296 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22adddad-6c0c-4608-9c53-45b0644d677a-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.531270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-cni-bin\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-var-lib-kubelet\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj8pd\" (UniqueName: \"kubernetes.io/projected/c6c24d13-14a9-4f94-8112-0ff9d1f62946-kube-api-access-qj8pd\") pod \"node-resolver-bp8js\" (UID: \"c6c24d13-14a9-4f94-8112-0ff9d1f62946\") " pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530384 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6c24d13-14a9-4f94-8112-0ff9d1f62946-hosts-file\") pod \"node-resolver-bp8js\" (UID: \"c6c24d13-14a9-4f94-8112-0ff9d1f62946\") " pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530409 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-cni-dir\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530436 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0-konnectivity-ca\") pod \"konnectivity-agent-bkvtn\" (UID: \"7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0\") " pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530465 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-run-ovn\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530475 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-cni-dir\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-sysconfig\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530556 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-sysconfig\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530581 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-var-lib-cni-bin\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530605 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-ovnkube-script-lib\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530618 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-var-lib-cni-bin\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-os-release\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-socket-dir-parent\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530695 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-run-k8s-cni-cncf-io\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530704 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-os-release\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-run-openvswitch\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530746 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-host-run-k8s-cni-cncf-io\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530775 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-node-log\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530862 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-socket-dir-parent\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530878 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530910 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-log-socket\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-systemd\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530946 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6c24d13-14a9-4f94-8112-0ff9d1f62946-tmp-dir\") pod \"node-resolver-bp8js\" (UID: \"c6c24d13-14a9-4f94-8112-0ff9d1f62946\") " pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ptjk\" (UniqueName: \"kubernetes.io/projected/22adddad-6c0c-4608-9c53-45b0644d677a-kube-api-access-5ptjk\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530994 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-systemd\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.530997 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-kubelet-config\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-dbus\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531060 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-slash\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531103 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-device-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.532588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9j6d\" (UniqueName: \"kubernetes.io/projected/5f4eec8c-1779-4a2f-b0e0-a028da53560b-kube-api-access-v9j6d\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-cni-netd\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-sys\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531241 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-daemon-config\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531248 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6c24d13-14a9-4f94-8112-0ff9d1f62946-tmp-dir\") pod \"node-resolver-bp8js\" (UID: \"c6c24d13-14a9-4f94-8112-0ff9d1f62946\") " pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531264 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-cnibin\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqbcc\" (UniqueName: \"kubernetes.io/projected/f5b26be1-08e7-4b34-a136-92a23404cb19-kube-api-access-vqbcc\") pod \"iptables-alerter-99x6w\" (UID: \"f5b26be1-08e7-4b34-a136-92a23404cb19\") " pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22adddad-6c0c-4608-9c53-45b0644d677a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531323 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22adddad-6c0c-4608-9c53-45b0644d677a-cnibin\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531341 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-var-lib-openvswitch\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531379 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-sys\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531696 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-multus-daemon-config\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.531930 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22adddad-6c0c-4608-9c53-45b0644d677a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.532250 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-etc-tuned\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.533167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.532499 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-tmp\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.537231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.537197 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nfw2\" (UniqueName: \"kubernetes.io/projected/a0e3e451-1eca-4d88-97b8-7aa14025cd9a-kube-api-access-7nfw2\") pod \"multus-srh2f\" (UID: \"a0e3e451-1eca-4d88-97b8-7aa14025cd9a\") " pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.539038 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.538307 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj8pd\" (UniqueName: \"kubernetes.io/projected/c6c24d13-14a9-4f94-8112-0ff9d1f62946-kube-api-access-qj8pd\") pod \"node-resolver-bp8js\" (UID: \"c6c24d13-14a9-4f94-8112-0ff9d1f62946\") " pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.539430 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.539407 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7n5b\" (UniqueName: \"kubernetes.io/projected/6cb5d18f-cc88-47b1-b95e-e03ac43dab2b-kube-api-access-f7n5b\") pod \"tuned-9dxx6\" (UID: \"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b\") " pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.539531 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.539468 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqbcc\" (UniqueName: \"kubernetes.io/projected/f5b26be1-08e7-4b34-a136-92a23404cb19-kube-api-access-vqbcc\") pod \"iptables-alerter-99x6w\" (UID: \"f5b26be1-08e7-4b34-a136-92a23404cb19\") " pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.539531 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.539475 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9j6d\" (UniqueName: \"kubernetes.io/projected/5f4eec8c-1779-4a2f-b0e0-a028da53560b-kube-api-access-v9j6d\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:55.539720 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.539700 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ptjk\" (UniqueName: \"kubernetes.io/projected/22adddad-6c0c-4608-9c53-45b0644d677a-kube-api-access-5ptjk\") pod \"multus-additional-cni-plugins-hwmq5\" (UID: \"22adddad-6c0c-4608-9c53-45b0644d677a\") " pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.631955 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.631920 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-cni-bin\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.631955 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.631954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0-konnectivity-ca\") pod \"konnectivity-agent-bkvtn\" (UID: \"7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0\") " pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.631970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-run-ovn\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.631985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-ovnkube-script-lib\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632034 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632048 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-run-ovn\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632051 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-cni-bin\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632063 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-run-openvswitch\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632096 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-run-openvswitch\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-node-log\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632142 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-log-socket\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632156 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-kubelet-config\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632174 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-node-log\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632194 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-log-socket\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-dbus\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-slash\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632230 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-kubelet-config\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632245 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-device-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632270 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-cni-netd\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632279 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-slash\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632300 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-var-lib-openvswitch\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632306 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-device-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-registration-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632347 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-var-lib-openvswitch\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632366 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-dbus\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632327 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-cni-netd\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0-agent-certs\") pod \"konnectivity-agent-bkvtn\" (UID: \"7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0\") " pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632390 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-registration-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632414 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-ovn-node-metrics-cert\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632468 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:02:55.632905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632506 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-socket-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632580 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0-konnectivity-ca\") pod \"konnectivity-agent-bkvtn\" (UID: \"7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0\") " pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632586 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9862602e-db24-4b4a-ab24-a6b16129404e-host\") pod \"node-ca-txkmb\" (UID: \"9862602e-db24-4b4a-ab24-a6b16129404e\") " pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632615 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-kubelet\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cj6t\" (UniqueName: \"kubernetes.io/projected/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-kube-api-access-6cj6t\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632674 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-socket-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-run-netns\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-sys-fs\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632719 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-kubelet\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxqq4\" (UniqueName: \"kubernetes.io/projected/987b4ea0-fe2b-48af-8c3e-608f7109cb57-kube-api-access-hxqq4\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632742 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632757 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-host-run-netns\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632728 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9862602e-db24-4b4a-ab24-a6b16129404e-host\") pod \"node-ca-txkmb\" (UID: \"9862602e-db24-4b4a-ab24-a6b16129404e\") " pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-run-systemd\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632795 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-env-overrides\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.633980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632825 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/987b4ea0-fe2b-48af-8c3e-608f7109cb57-sys-fs\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9862602e-db24-4b4a-ab24-a6b16129404e-serviceca\") pod \"node-ca-txkmb\" (UID: \"9862602e-db24-4b4a-ab24-a6b16129404e\") " pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-ovnkube-config\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.632908 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-run-systemd\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.632974 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.633036 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret podName:db6d6e2f-7de5-4121-b6ca-d7f5aa168471 nodeName:}" failed. No retries permitted until 2026-06-06 22:02:56.133019406 +0000 UTC m=+3.131998882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret") pod "global-pull-secret-syncer-m7gsp" (UID: "db6d6e2f-7de5-4121-b6ca-d7f5aa168471") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.633179 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpj6z\" (UniqueName: \"kubernetes.io/projected/9862602e-db24-4b4a-ab24-a6b16129404e-kube-api-access-mpj6z\") pod \"node-ca-txkmb\" (UID: \"9862602e-db24-4b4a-ab24-a6b16129404e\") " pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.633195 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-env-overrides\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.633211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-systemd-units\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.633238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-etc-openvswitch\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.633306 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-etc-openvswitch\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.633332 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-ovnkube-script-lib\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.633347 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-systemd-units\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.633393 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-ovnkube-config\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.634831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.633413 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9862602e-db24-4b4a-ab24-a6b16129404e-serviceca\") pod \"node-ca-txkmb\" (UID: \"9862602e-db24-4b4a-ab24-a6b16129404e\") " pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.635404 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.635093 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-ovn-node-metrics-cert\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.635404 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.635233 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0-agent-certs\") pod \"konnectivity-agent-bkvtn\" (UID: \"7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0\") " pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:02:55.638870 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.638851 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:02:55.638977 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.638875 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:02:55.638977 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.638889 2580 projected.go:196] Error preparing data for projected volume kube-api-access-hlsq6 for pod openshift-network-diagnostics/network-check-target-vlk2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:02:55.638977 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:55.638960 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6 podName:50ef0dd1-9d49-4216-8151-55e7a7fffd8b nodeName:}" failed. No retries permitted until 2026-06-06 22:02:56.138940539 +0000 UTC m=+3.137920015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hlsq6" (UniqueName: "kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6") pod "network-check-target-vlk2c" (UID: "50ef0dd1-9d49-4216-8151-55e7a7fffd8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:02:55.641269 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.641251 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxqq4\" (UniqueName: \"kubernetes.io/projected/987b4ea0-fe2b-48af-8c3e-608f7109cb57-kube-api-access-hxqq4\") pod \"aws-ebs-csi-driver-node-zmk2k\" (UID: \"987b4ea0-fe2b-48af-8c3e-608f7109cb57\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.641640 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.641621 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cj6t\" (UniqueName: \"kubernetes.io/projected/c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa-kube-api-access-6cj6t\") pod \"ovnkube-node-74znx\" (UID: \"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.641705 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.641627 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpj6z\" (UniqueName: \"kubernetes.io/projected/9862602e-db24-4b4a-ab24-a6b16129404e-kube-api-access-mpj6z\") pod \"node-ca-txkmb\" (UID: \"9862602e-db24-4b4a-ab24-a6b16129404e\") " pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.718085 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.718048 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-srh2f" Jun 06 22:02:55.728647 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.728625 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" Jun 06 22:02:55.736373 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.736349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" Jun 06 22:02:55.742018 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.741992 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bp8js" Jun 06 22:02:55.749484 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.749462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-99x6w" Jun 06 22:02:55.757180 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.757160 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:02:55.763777 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.763755 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:02:55.771277 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.771233 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" Jun 06 22:02:55.776644 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.776629 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-txkmb" Jun 06 22:02:55.906405 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:55.906373 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:02:56.037404 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.037317 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:56.037555 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:56.037473 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:02:56.037619 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:56.037561 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs podName:5f4eec8c-1779-4a2f-b0e0-a028da53560b nodeName:}" failed. No retries permitted until 2026-06-06 22:02:57.037541721 +0000 UTC m=+4.036521184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs") pod "network-metrics-daemon-j7ksf" (UID: "5f4eec8c-1779-4a2f-b0e0-a028da53560b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:02:56.138557 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.138524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:56.138701 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:56.138627 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:02:56.138701 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:56.138695 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret podName:db6d6e2f-7de5-4121-b6ca-d7f5aa168471 nodeName:}" failed. No retries permitted until 2026-06-06 22:02:57.138675483 +0000 UTC m=+4.137654946 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret") pod "global-pull-secret-syncer-m7gsp" (UID: "db6d6e2f-7de5-4121-b6ca-d7f5aa168471") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:02:56.211570 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:56.211545 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9862602e_db24_4b4a_ab24_a6b16129404e.slice/crio-67ce64de44f2a8dcb11de9753f2b9a508aea090ea4b3c07c39825b3bbaa0236f WatchSource:0}: Error finding container 67ce64de44f2a8dcb11de9753f2b9a508aea090ea4b3c07c39825b3bbaa0236f: Status 404 returned error can't find the container with id 67ce64de44f2a8dcb11de9753f2b9a508aea090ea4b3c07c39825b3bbaa0236f Jun 06 22:02:56.220279 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:56.220255 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ca3dc0_5c16_4fa6_ae87_cf4e0fd060fa.slice/crio-366136dfd233c7cb866dd87973baca2c474ca28c484053d1b2bb0034cbc3f078 WatchSource:0}: Error finding container 366136dfd233c7cb866dd87973baca2c474ca28c484053d1b2bb0034cbc3f078: Status 404 returned error can't find the container with id 366136dfd233c7cb866dd87973baca2c474ca28c484053d1b2bb0034cbc3f078 Jun 06 22:02:56.221081 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:56.221061 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e3e451_1eca_4d88_97b8_7aa14025cd9a.slice/crio-7ca48c4af79ea881ef69507d8923c099cbcf4571da881316f9c1d067a9108629 WatchSource:0}: Error finding container 7ca48c4af79ea881ef69507d8923c099cbcf4571da881316f9c1d067a9108629: Status 404 returned error can't find the container with id 7ca48c4af79ea881ef69507d8923c099cbcf4571da881316f9c1d067a9108629 Jun 06 22:02:56.239268 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.239245 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:02:56.239354 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:56.239344 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:02:56.239399 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:56.239358 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:02:56.239399 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:56.239367 2580 projected.go:196] Error preparing data for projected volume kube-api-access-hlsq6 for pod openshift-network-diagnostics/network-check-target-vlk2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:02:56.239462 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:56.239406 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6 podName:50ef0dd1-9d49-4216-8151-55e7a7fffd8b nodeName:}" failed. No retries permitted until 2026-06-06 22:02:57.23939473 +0000 UTC m=+4.238374191 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hlsq6" (UniqueName: "kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6") pod "network-check-target-vlk2c" (UID: "50ef0dd1-9d49-4216-8151-55e7a7fffd8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:02:56.242137 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:56.242117 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff591cd_bfd1_4f0e_8bc3_4a42011a49b0.slice/crio-02ec4b92a60d941b8f406efe191e9f40d92925890738849f510d82618fcf689b WatchSource:0}: Error finding container 02ec4b92a60d941b8f406efe191e9f40d92925890738849f510d82618fcf689b: Status 404 returned error can't find the container with id 02ec4b92a60d941b8f406efe191e9f40d92925890738849f510d82618fcf689b Jun 06 22:02:56.244584 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:56.244559 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb5d18f_cc88_47b1_b95e_e03ac43dab2b.slice/crio-500b80de1a117b95d91bcde9fe433b281d6ea09d56d8601b06304c6f48fa3b21 WatchSource:0}: Error finding container 500b80de1a117b95d91bcde9fe433b281d6ea09d56d8601b06304c6f48fa3b21: Status 404 returned error can't find the container with id 500b80de1a117b95d91bcde9fe433b281d6ea09d56d8601b06304c6f48fa3b21 Jun 06 22:02:56.245662 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:56.245620 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22adddad_6c0c_4608_9c53_45b0644d677a.slice/crio-2759053c7c5588d0fd46fa293e64990f1de9b43cc9ff92d30b21170ba9be9945 WatchSource:0}: Error finding container 2759053c7c5588d0fd46fa293e64990f1de9b43cc9ff92d30b21170ba9be9945: Status 404 returned error can't find the container with id 2759053c7c5588d0fd46fa293e64990f1de9b43cc9ff92d30b21170ba9be9945 Jun 06 22:02:56.246243 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:02:56.246224 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b26be1_08e7_4b34_a136_92a23404cb19.slice/crio-7feed80157d101ad967d5fa4e26eccca9dae8c6e52035ba694be46b0be6ac5f5 WatchSource:0}: Error finding container 7feed80157d101ad967d5fa4e26eccca9dae8c6e52035ba694be46b0be6ac5f5: Status 404 returned error can't find the container with id 7feed80157d101ad967d5fa4e26eccca9dae8c6e52035ba694be46b0be6ac5f5 Jun 06 22:02:56.458232 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.458170 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-05 21:57:55 +0000 UTC" deadline="2028-02-07 18:25:03.395059718 +0000 UTC" Jun 06 22:02:56.458232 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.458228 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14660h22m6.936835638s" Jun 06 22:02:56.482511 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.482486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:02:56.482626 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:56.482579 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:02:56.491111 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.491079 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-99x6w" event={"ID":"f5b26be1-08e7-4b34-a136-92a23404cb19","Type":"ContainerStarted","Data":"7feed80157d101ad967d5fa4e26eccca9dae8c6e52035ba694be46b0be6ac5f5"} Jun 06 22:02:56.492155 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.492130 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bp8js" event={"ID":"c6c24d13-14a9-4f94-8112-0ff9d1f62946","Type":"ContainerStarted","Data":"b48fcfa3e7ef08c183c223eb2aea355dee971643ad80184c89d40908bd3ffba7"} Jun 06 22:02:56.493223 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.493195 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" event={"ID":"987b4ea0-fe2b-48af-8c3e-608f7109cb57","Type":"ContainerStarted","Data":"db7e68036071f2540be1aedb67b6c2f17cb08a5af69c2c03f4c817a3059b08a9"} Jun 06 22:02:56.494734 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.494711 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" event={"ID":"22adddad-6c0c-4608-9c53-45b0644d677a","Type":"ContainerStarted","Data":"2759053c7c5588d0fd46fa293e64990f1de9b43cc9ff92d30b21170ba9be9945"} Jun 06 22:02:56.495789 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.495755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" event={"ID":"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b","Type":"ContainerStarted","Data":"500b80de1a117b95d91bcde9fe433b281d6ea09d56d8601b06304c6f48fa3b21"} Jun 06 22:02:56.496910 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.496883 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bkvtn" event={"ID":"7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0","Type":"ContainerStarted","Data":"02ec4b92a60d941b8f406efe191e9f40d92925890738849f510d82618fcf689b"} Jun 06 22:02:56.497958 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.497937 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-srh2f" event={"ID":"a0e3e451-1eca-4d88-97b8-7aa14025cd9a","Type":"ContainerStarted","Data":"7ca48c4af79ea881ef69507d8923c099cbcf4571da881316f9c1d067a9108629"} Jun 06 22:02:56.499026 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.499004 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerStarted","Data":"366136dfd233c7cb866dd87973baca2c474ca28c484053d1b2bb0034cbc3f078"} Jun 06 22:02:56.500276 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.500256 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-txkmb" event={"ID":"9862602e-db24-4b4a-ab24-a6b16129404e","Type":"ContainerStarted","Data":"67ce64de44f2a8dcb11de9753f2b9a508aea090ea4b3c07c39825b3bbaa0236f"} Jun 06 22:02:56.502100 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.502072 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal" event={"ID":"2f312d8097474ef9a1a11d8e634ff7d8","Type":"ContainerStarted","Data":"ff584310ab51f944716e361df3047cdc49736c609482dc903c6174b4ee81fff0"} Jun 06 22:02:56.509563 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.509541 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 06 22:02:56.518610 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:56.518573 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-115.ec2.internal" podStartSLOduration=2.518562538 podStartE2EDuration="2.518562538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:02:56.518234796 +0000 UTC m=+3.517214278" watchObservedRunningTime="2026-06-06 22:02:56.518562538 +0000 UTC m=+3.517542030" Jun 06 22:02:57.046153 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:57.046116 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:57.046330 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.046270 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:02:57.046403 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.046335 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs podName:5f4eec8c-1779-4a2f-b0e0-a028da53560b nodeName:}" failed. No retries permitted until 2026-06-06 22:02:59.046316216 +0000 UTC m=+6.045295682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs") pod "network-metrics-daemon-j7ksf" (UID: "5f4eec8c-1779-4a2f-b0e0-a028da53560b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:02:57.147535 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:57.146945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:57.147535 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.147085 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:02:57.147535 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.147151 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret podName:db6d6e2f-7de5-4121-b6ca-d7f5aa168471 nodeName:}" failed. No retries permitted until 2026-06-06 22:02:59.147131993 +0000 UTC m=+6.146111460 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret") pod "global-pull-secret-syncer-m7gsp" (UID: "db6d6e2f-7de5-4121-b6ca-d7f5aa168471") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:02:57.248094 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:57.248058 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:02:57.248296 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.248274 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:02:57.248363 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.248300 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:02:57.248363 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.248314 2580 projected.go:196] Error preparing data for projected volume kube-api-access-hlsq6 for pod openshift-network-diagnostics/network-check-target-vlk2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:02:57.248467 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.248369 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6 podName:50ef0dd1-9d49-4216-8151-55e7a7fffd8b nodeName:}" failed. No retries permitted until 2026-06-06 22:02:59.248351775 +0000 UTC m=+6.247331242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hlsq6" (UniqueName: "kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6") pod "network-check-target-vlk2c" (UID: "50ef0dd1-9d49-4216-8151-55e7a7fffd8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:02:57.459135 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:57.459010 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-05 21:57:55 +0000 UTC" deadline="2027-12-18 07:18:32.854280282 +0000 UTC" Jun 06 22:02:57.459135 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:57.459048 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13425h15m35.395237309s" Jun 06 22:02:57.484630 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:57.483845 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:57.484630 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:57.484012 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:57.484630 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.484145 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:02:57.484630 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:57.484562 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:02:57.512830 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:57.512452 2580 generic.go:356] "Generic (PLEG): container finished" podID="809d1195e782cf85c5467f6d560ad47d" containerID="e8369f399c3a642edec15668a5f4890ece54ab24fe604e61ede163e1c2ab5ced" exitCode=0 Jun 06 22:02:57.513077 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:57.513013 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" event={"ID":"809d1195e782cf85c5467f6d560ad47d","Type":"ContainerDied","Data":"e8369f399c3a642edec15668a5f4890ece54ab24fe604e61ede163e1c2ab5ced"} Jun 06 22:02:58.482749 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:58.482712 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:02:58.483272 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:58.482850 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:02:58.549477 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:58.549434 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" event={"ID":"809d1195e782cf85c5467f6d560ad47d","Type":"ContainerStarted","Data":"d4651c937da0e13bcfb465e17a214a01aa44faa4988ea0911bff9f10050d39c5"} Jun 06 22:02:58.568722 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:58.567884 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-115.ec2.internal" podStartSLOduration=4.567862144 podStartE2EDuration="4.567862144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:02:58.567397554 +0000 UTC m=+5.566377039" watchObservedRunningTime="2026-06-06 22:02:58.567862144 +0000 UTC m=+5.566841629" Jun 06 22:02:59.064498 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:59.064464 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:59.064700 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.064622 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:02:59.064700 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.064685 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs podName:5f4eec8c-1779-4a2f-b0e0-a028da53560b nodeName:}" failed. No retries permitted until 2026-06-06 22:03:03.064667264 +0000 UTC m=+10.063646740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs") pod "network-metrics-daemon-j7ksf" (UID: "5f4eec8c-1779-4a2f-b0e0-a028da53560b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:02:59.165630 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:59.165590 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:59.165796 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.165743 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:02:59.165901 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.165869 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret podName:db6d6e2f-7de5-4121-b6ca-d7f5aa168471 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:03.165786346 +0000 UTC m=+10.164765827 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret") pod "global-pull-secret-syncer-m7gsp" (UID: "db6d6e2f-7de5-4121-b6ca-d7f5aa168471") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:02:59.265987 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:59.265948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:02:59.266162 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.266141 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:02:59.266162 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.266159 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:02:59.266269 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.266171 2580 projected.go:196] Error preparing data for projected volume kube-api-access-hlsq6 for pod openshift-network-diagnostics/network-check-target-vlk2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:02:59.266269 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.266230 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6 podName:50ef0dd1-9d49-4216-8151-55e7a7fffd8b nodeName:}" failed. No retries permitted until 2026-06-06 22:03:03.266211641 +0000 UTC m=+10.265191123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hlsq6" (UniqueName: "kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6") pod "network-check-target-vlk2c" (UID: "50ef0dd1-9d49-4216-8151-55e7a7fffd8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:02:59.482819 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:59.482778 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:02:59.483406 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:02:59.482788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:02:59.483406 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.482917 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:02:59.483406 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:02:59.483014 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:00.482977 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:00.482939 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:00.483456 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:00.483084 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:01.482933 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:01.482896 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:01.483116 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:01.483094 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:01.483116 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:01.483096 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:01.483530 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:01.483215 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:02.482788 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:02.482753 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:02.482985 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:02.482902 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:03.098745 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:03.098708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:03.099239 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.098879 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:03.099239 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.098964 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs podName:5f4eec8c-1779-4a2f-b0e0-a028da53560b nodeName:}" failed. No retries permitted until 2026-06-06 22:03:11.098941982 +0000 UTC m=+18.097921767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs") pod "network-metrics-daemon-j7ksf" (UID: "5f4eec8c-1779-4a2f-b0e0-a028da53560b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:03.199919 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:03.199876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:03.200100 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.200029 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:03.200100 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.200081 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret podName:db6d6e2f-7de5-4121-b6ca-d7f5aa168471 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:11.200066027 +0000 UTC m=+18.199045489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret") pod "global-pull-secret-syncer-m7gsp" (UID: "db6d6e2f-7de5-4121-b6ca-d7f5aa168471") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:03.301080 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:03.300978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:03.301278 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.301132 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:03.301278 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.301163 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:03.301278 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.301181 2580 projected.go:196] Error preparing data for projected volume kube-api-access-hlsq6 for pod openshift-network-diagnostics/network-check-target-vlk2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:03.301278 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.301250 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6 podName:50ef0dd1-9d49-4216-8151-55e7a7fffd8b nodeName:}" failed. No retries permitted until 2026-06-06 22:03:11.301229556 +0000 UTC m=+18.300209041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hlsq6" (UniqueName: "kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6") pod "network-check-target-vlk2c" (UID: "50ef0dd1-9d49-4216-8151-55e7a7fffd8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:03.487526 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:03.487500 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:03.487784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:03.487578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:03.487952 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.487929 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:03.488303 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:03.488260 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:04.483337 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:04.483297 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:04.483792 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:04.483423 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:05.482579 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:05.482518 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:05.482759 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:05.482655 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:05.482759 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:05.482711 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:05.482905 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:05.482840 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:06.483197 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:06.483122 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:06.483656 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:06.483249 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:07.483076 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:07.483033 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:07.483076 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:07.483055 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:07.483556 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:07.483159 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:07.483556 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:07.483277 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:08.483081 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:08.483053 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:08.483231 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:08.483164 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:09.486447 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:09.486415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:09.486447 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:09.486456 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:09.486982 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:09.486541 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:09.486982 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:09.486672 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:10.483183 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:10.483151 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:10.483467 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:10.483275 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:11.161461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:11.161429 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:11.161894 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.161553 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:11.161894 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.161609 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs podName:5f4eec8c-1779-4a2f-b0e0-a028da53560b nodeName:}" failed. No retries permitted until 2026-06-06 22:03:27.161595317 +0000 UTC m=+34.160574779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs") pod "network-metrics-daemon-j7ksf" (UID: "5f4eec8c-1779-4a2f-b0e0-a028da53560b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:11.262337 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:11.262298 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:11.262532 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.262425 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:11.262532 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.262513 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret podName:db6d6e2f-7de5-4121-b6ca-d7f5aa168471 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:27.262491556 +0000 UTC m=+34.261471041 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret") pod "global-pull-secret-syncer-m7gsp" (UID: "db6d6e2f-7de5-4121-b6ca-d7f5aa168471") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:11.362818 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:11.362767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:11.362997 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.362932 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:11.362997 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.362954 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:11.362997 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.362971 2580 projected.go:196] Error preparing data for projected volume kube-api-access-hlsq6 for pod openshift-network-diagnostics/network-check-target-vlk2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:11.363145 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.363033 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6 podName:50ef0dd1-9d49-4216-8151-55e7a7fffd8b nodeName:}" failed. No retries permitted until 2026-06-06 22:03:27.363015628 +0000 UTC m=+34.361995089 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hlsq6" (UniqueName: "kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6") pod "network-check-target-vlk2c" (UID: "50ef0dd1-9d49-4216-8151-55e7a7fffd8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:11.486539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:11.486465 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:11.486689 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.486588 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:11.486689 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:11.486644 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:11.486793 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:11.486739 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:12.482906 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:12.482874 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:12.483382 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:12.482989 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:13.484270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:13.484228 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:13.484737 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:13.484346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:13.484737 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:13.484432 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:13.484737 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:13.484475 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:14.482691 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:14.482654 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:14.482958 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:14.482792 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:15.482673 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:15.482646 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:15.483084 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:15.482783 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:15.483232 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:15.483206 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:15.483354 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:15.483334 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:15.583961 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:15.583934 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bkvtn" event={"ID":"7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0","Type":"ContainerStarted","Data":"9d459599ca7e6279cfa02aa8eb9a2cf2fa47be4942e90c626d79eb4a9ebf93eb"} Jun 06 22:03:15.588998 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:15.588970 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bp8js" event={"ID":"c6c24d13-14a9-4f94-8112-0ff9d1f62946","Type":"ContainerStarted","Data":"f8b97faacd0d1e5def26a7c8fd02b1534a861bde960fbb7f8fbe3c2b21dad5f5"} Jun 06 22:03:15.602663 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:15.602482 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bkvtn" podStartSLOduration=3.8129878919999998 podStartE2EDuration="22.602463663s" totalImagesPullingTime="18.789475771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:15.601923768 +0000 UTC m=+22.600903253" watchObservedRunningTime="2026-06-06 22:03:15.602463663 +0000 UTC m=+22.601443159" Jun 06 22:03:15.623949 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:15.623838 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bp8js" podStartSLOduration=3.809367461 podStartE2EDuration="22.623820333s" totalImagesPullingTime="18.814452872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:15.62224984 +0000 UTC m=+22.621229327" watchObservedRunningTime="2026-06-06 22:03:15.623820333 +0000 UTC m=+22.622799808" Jun 06 22:03:16.135304 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.135283 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:03:16.483042 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.483010 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:16.483896 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:16.483118 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:16.592606 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.592572 2580 generic.go:356] "Generic (PLEG): container finished" podID="22adddad-6c0c-4608-9c53-45b0644d677a" containerID="c9ac84f3c45ad54378c7a1f0b89f9a1fab3f46aec11e59eadd99f78382ef3cdb" exitCode=0 Jun 06 22:03:16.592788 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.592659 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" event={"ID":"22adddad-6c0c-4608-9c53-45b0644d677a","Type":"ContainerDied","Data":"c9ac84f3c45ad54378c7a1f0b89f9a1fab3f46aec11e59eadd99f78382ef3cdb"} Jun 06 22:03:16.594106 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.594092 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" event={"ID":"6cb5d18f-cc88-47b1-b95e-e03ac43dab2b","Type":"ContainerStarted","Data":"4f17c7c80dcffc36f2a9759f47656f23588ce4288d841aab975599593b9bb89f"} Jun 06 22:03:16.595589 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.595569 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-srh2f" event={"ID":"a0e3e451-1eca-4d88-97b8-7aa14025cd9a","Type":"ContainerStarted","Data":"07c447fe03b74c7c55163787d7a6633b299c74e8e9b3c2751fcb1dd9ced5f466"} Jun 06 22:03:16.598257 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.598232 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:03:16.598581 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.598563 2580 generic.go:356] "Generic (PLEG): container finished" podID="c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa" containerID="d18153434b5fb4dc19d7701e856cae38d0ef315fad372b9367833f8c82a1512e" exitCode=1 Jun 06 22:03:16.598639 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.598624 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerStarted","Data":"c314656fb8893435b01d48056c6a7634851c557c4d5c2a159f3a5f44821c2294"} Jun 06 22:03:16.598684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.598674 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerStarted","Data":"4110b0e025656fd62338885e8b733b048a82602b60e59f70b2cbf50faaea1410"} Jun 06 22:03:16.598718 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.598708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerStarted","Data":"7f6764724e8edac6e549ad96a69f3569ffa2ff933cc144ccf9ebb6d88077aca9"} Jun 06 22:03:16.598750 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.598738 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerStarted","Data":"e50d6b4d3742a2a43ac97f21487c4f8f9bd07f370b8523b46908b667fd9aecb4"} Jun 06 22:03:16.598781 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.598773 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerDied","Data":"d18153434b5fb4dc19d7701e856cae38d0ef315fad372b9367833f8c82a1512e"} Jun 06 22:03:16.598840 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.598829 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerStarted","Data":"de95e9f35e06a742705aa4b33c9a94e293044388c69b6063cca4750dd4c9cb2a"} Jun 06 22:03:16.600330 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.600311 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-txkmb" event={"ID":"9862602e-db24-4b4a-ab24-a6b16129404e","Type":"ContainerStarted","Data":"5cbeba8fad73e9d1117630a3348a49965d8893a999519b242540e1d31eeec64a"} Jun 06 22:03:16.601553 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.601523 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" event={"ID":"987b4ea0-fe2b-48af-8c3e-608f7109cb57","Type":"ContainerStarted","Data":"a0e81c8b4cdead574c2c01074826ef78d560bdd95a7debd8f6b10cc7b2d9511f"} Jun 06 22:03:16.650154 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.650115 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-txkmb" podStartSLOduration=4.453221312 podStartE2EDuration="23.650100083s" totalImagesPullingTime="19.196878771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:16.632504077 +0000 UTC m=+23.631483561" watchObservedRunningTime="2026-06-06 22:03:16.650100083 +0000 UTC m=+23.649079569" Jun 06 22:03:16.664278 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.664237 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9dxx6" podStartSLOduration=4.502168861 podStartE2EDuration="23.664228081s" totalImagesPullingTime="19.16205922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:16.664217427 +0000 UTC m=+23.663196910" watchObservedRunningTime="2026-06-06 22:03:16.664228081 +0000 UTC m=+23.663207565" Jun 06 22:03:16.664435 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:16.664411 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-srh2f" podStartSLOduration=4.480343201 podStartE2EDuration="23.664405474s" totalImagesPullingTime="19.184062273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:16.649877814 +0000 UTC m=+23.648857308" watchObservedRunningTime="2026-06-06 22:03:16.664405474 +0000 UTC m=+23.663384957" Jun 06 22:03:17.055357 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.055330 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 06 22:03:17.482765 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.482561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:17.483072 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.482561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:17.483072 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:17.482892 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:17.483072 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:17.482962 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:17.485664 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.485584 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-06T22:03:17.055351143Z","UUID":"c5c1a075-0f7f-43a6-8c13-033efbee49b7","Handler":null,"Name":"","Endpoint":""} Jun 06 22:03:17.487622 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.487524 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 06 22:03:17.487622 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.487564 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 06 22:03:17.605822 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.605758 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-99x6w" event={"ID":"f5b26be1-08e7-4b34-a136-92a23404cb19","Type":"ContainerStarted","Data":"4decec8f77f0e1128411f19912a03e3fef9c3e15d12cea6b60a95db9c3a9cdbf"} Jun 06 22:03:17.608411 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.608377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" event={"ID":"987b4ea0-fe2b-48af-8c3e-608f7109cb57","Type":"ContainerStarted","Data":"6a4f5436cc902efe71f40f905a784cf98c702e242d45af013089fb8f2a996cc8"} Jun 06 22:03:17.623574 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.623521 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-99x6w" podStartSLOduration=5.460305843 podStartE2EDuration="24.623501328s" totalImagesPullingTime="19.163195485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:17.622735002 +0000 UTC m=+24.621714486" watchObservedRunningTime="2026-06-06 22:03:17.623501328 +0000 UTC m=+24.622480813" Jun 06 22:03:17.927375 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.927346 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:03:17.928070 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:17.928048 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:03:18.483230 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:18.483202 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:18.483740 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:18.483325 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:18.613088 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:18.613048 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" event={"ID":"987b4ea0-fe2b-48af-8c3e-608f7109cb57","Type":"ContainerStarted","Data":"55ff320653693094da3863588f4e205a254a4c06c41d4d1aa01b8ade04bf5470"} Jun 06 22:03:18.616591 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:18.616567 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:03:18.617025 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:18.617000 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerStarted","Data":"0073ddbe4ccea9d65e932de5188cbb30afff700f0f730850dddffb2757440aa2"} Jun 06 22:03:18.617796 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:18.617776 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bkvtn" Jun 06 22:03:18.630259 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:18.630221 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmk2k" podStartSLOduration=4.153623761 podStartE2EDuration="25.630209191s" totalImagesPullingTime="21.47658543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:18.629844895 +0000 UTC m=+25.628824379" watchObservedRunningTime="2026-06-06 22:03:18.630209191 +0000 UTC m=+25.629188674" Jun 06 22:03:19.482569 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:19.482534 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:19.482860 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:19.482677 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:19.482860 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:19.482729 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:19.482980 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:19.482868 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:20.482877 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:20.482838 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:20.483618 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:20.482978 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:20.625427 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:20.625085 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:03:20.626107 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:20.626028 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerStarted","Data":"c4e049ef72293c73ed0242cffc59bff1b6cad1d592db7078cb24268d09ca61fa"} Jun 06 22:03:20.626684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:20.626476 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:03:20.626684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:20.626558 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:03:20.626684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:20.626607 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:03:20.626684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:20.626478 2580 scope.go:117] "RemoveContainer" containerID="d18153434b5fb4dc19d7701e856cae38d0ef315fad372b9367833f8c82a1512e" Jun 06 22:03:20.643070 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:20.643002 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:03:20.646918 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:20.646481 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:03:21.482893 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:21.482796 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:21.483459 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:21.482928 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:21.483459 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:21.482979 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:21.483459 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:21.483369 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:21.631136 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:21.631109 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:03:21.631502 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:21.631467 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" event={"ID":"c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa","Type":"ContainerStarted","Data":"011d6b53afc0da8f493313ae6be87b929b5721b3d97031ad1f64c1c48739dda5"} Jun 06 22:03:21.633390 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:21.633367 2580 generic.go:356] "Generic (PLEG): container finished" podID="22adddad-6c0c-4608-9c53-45b0644d677a" containerID="98d1b94dcc02bf0602afd4bbc02caa04d1715d30a88c192dd1b4a862ee0f9286" exitCode=0 Jun 06 22:03:21.633475 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:21.633400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" event={"ID":"22adddad-6c0c-4608-9c53-45b0644d677a","Type":"ContainerDied","Data":"98d1b94dcc02bf0602afd4bbc02caa04d1715d30a88c192dd1b4a862ee0f9286"} Jun 06 22:03:21.658067 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:21.658022 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" podStartSLOduration=9.311495694 podStartE2EDuration="28.658009647s" totalImagesPullingTime="19.346513953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:21.656339055 +0000 UTC m=+28.655318538" watchObservedRunningTime="2026-06-06 22:03:21.658009647 +0000 UTC m=+28.656989131" Jun 06 22:03:22.483228 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:22.483064 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:22.483670 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:22.483298 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:22.534175 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:22.534145 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vlk2c"] Jun 06 22:03:22.534859 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:22.534838 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-m7gsp"] Jun 06 22:03:22.534993 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:22.534953 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:22.535080 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:22.535058 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:22.535544 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:22.535526 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j7ksf"] Jun 06 22:03:22.535650 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:22.535626 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:22.535753 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:22.535735 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:22.638324 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:22.638293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" event={"ID":"22adddad-6c0c-4608-9c53-45b0644d677a","Type":"ContainerStarted","Data":"f2e9d4bb34bcfbbee423f89a38f51e4de1f59522273b888cdbda99d954f9fb80"} Jun 06 22:03:22.638855 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:22.638837 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:22.638952 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:22.638916 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:23.641982 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:23.641930 2580 generic.go:356] "Generic (PLEG): container finished" podID="22adddad-6c0c-4608-9c53-45b0644d677a" containerID="f2e9d4bb34bcfbbee423f89a38f51e4de1f59522273b888cdbda99d954f9fb80" exitCode=0 Jun 06 22:03:23.642397 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:23.642001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" event={"ID":"22adddad-6c0c-4608-9c53-45b0644d677a","Type":"ContainerDied","Data":"f2e9d4bb34bcfbbee423f89a38f51e4de1f59522273b888cdbda99d954f9fb80"} Jun 06 22:03:24.483181 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:24.483154 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:24.483348 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:24.483151 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:24.483348 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:24.483260 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:24.483348 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:24.483280 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:24.483462 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:24.483356 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:24.483462 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:24.483414 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:25.648989 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:25.648958 2580 generic.go:356] "Generic (PLEG): container finished" podID="22adddad-6c0c-4608-9c53-45b0644d677a" containerID="e56dafed606fd9ce033b384bb3437bc62fdc270a4fe07f17936c1ac02586b9d0" exitCode=0 Jun 06 22:03:25.649554 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:25.649040 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" event={"ID":"22adddad-6c0c-4608-9c53-45b0644d677a","Type":"ContainerDied","Data":"e56dafed606fd9ce033b384bb3437bc62fdc270a4fe07f17936c1ac02586b9d0"} Jun 06 22:03:26.483205 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:26.483164 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:26.483205 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:26.483205 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:26.483468 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:26.483294 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:26.483468 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:26.483309 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gsp" podUID="db6d6e2f-7de5-4121-b6ca-d7f5aa168471" Jun 06 22:03:26.483468 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:26.483424 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:03:26.483627 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:26.483513 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlk2c" podUID="50ef0dd1-9d49-4216-8151-55e7a7fffd8b" Jun 06 22:03:27.176488 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:27.176450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:27.176912 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:27.176623 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:27.176912 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:27.176703 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs podName:5f4eec8c-1779-4a2f-b0e0-a028da53560b nodeName:}" failed. No retries permitted until 2026-06-06 22:03:59.176684551 +0000 UTC m=+66.175664013 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs") pod "network-metrics-daemon-j7ksf" (UID: "5f4eec8c-1779-4a2f-b0e0-a028da53560b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 06 22:03:27.277012 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:27.276975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:27.277192 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:27.277101 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:27.277192 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:27.277166 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret podName:db6d6e2f-7de5-4121-b6ca-d7f5aa168471 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:59.277149071 +0000 UTC m=+66.276128548 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret") pod "global-pull-secret-syncer-m7gsp" (UID: "db6d6e2f-7de5-4121-b6ca-d7f5aa168471") : object "kube-system"/"original-pull-secret" not registered Jun 06 22:03:27.377502 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:27.377467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:27.377670 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:27.377620 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 06 22:03:27.377670 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:27.377637 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 06 22:03:27.377670 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:27.377648 2580 projected.go:196] Error preparing data for projected volume kube-api-access-hlsq6 for pod openshift-network-diagnostics/network-check-target-vlk2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:27.377833 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:27.377697 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6 podName:50ef0dd1-9d49-4216-8151-55e7a7fffd8b nodeName:}" failed. No retries permitted until 2026-06-06 22:03:59.377681687 +0000 UTC m=+66.376661148 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hlsq6" (UniqueName: "kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6") pod "network-check-target-vlk2c" (UID: "50ef0dd1-9d49-4216-8151-55e7a7fffd8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 06 22:03:28.345040 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.344962 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-115.ec2.internal" event="NodeReady" Jun 06 22:03:28.345470 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.345113 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 06 22:03:28.399475 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.399445 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ld6pc"] Jun 06 22:03:28.401938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.401893 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.402084 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.401903 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n5r5h"] Jun 06 22:03:28.403671 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.403647 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n5r5h" Jun 06 22:03:28.404488 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.404466 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 06 22:03:28.404606 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.404548 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 06 22:03:28.404670 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.404648 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j597l\"" Jun 06 22:03:28.406727 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.406710 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 06 22:03:28.406846 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.406821 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nsrrs\"" Jun 06 22:03:28.407250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.407232 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 06 22:03:28.407333 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.407318 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 06 22:03:28.416703 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.416678 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n5r5h"] Jun 06 22:03:28.444786 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.444754 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ld6pc"] Jun 06 22:03:28.482608 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.482581 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:28.482763 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.482620 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:28.482763 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.482688 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:28.485101 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.485237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485112 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/89a055b7-3e1a-4f23-85cf-e60006d08741-config-volume\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.485237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mn4r\" (UniqueName: \"kubernetes.io/projected/89a055b7-3e1a-4f23-85cf-e60006d08741-kube-api-access-7mn4r\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.485237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89a055b7-3e1a-4f23-85cf-e60006d08741-tmp-dir\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.485419 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51877535-3dba-451c-912e-38b3b31cab69-cert\") pod \"ingress-canary-n5r5h\" (UID: \"51877535-3dba-451c-912e-38b3b31cab69\") " pod="openshift-ingress-canary/ingress-canary-n5r5h" Jun 06 22:03:28.485419 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9qp6\" (UniqueName: \"kubernetes.io/projected/51877535-3dba-451c-912e-38b3b31cab69-kube-api-access-b9qp6\") pod \"ingress-canary-n5r5h\" (UID: \"51877535-3dba-451c-912e-38b3b31cab69\") " pod="openshift-ingress-canary/ingress-canary-n5r5h" Jun 06 22:03:28.485673 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485651 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 06 22:03:28.485673 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485665 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 06 22:03:28.485865 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485658 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 06 22:03:28.485865 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485661 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ksb8v\"" Jun 06 22:03:28.485865 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485659 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 06 22:03:28.485865 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.485781 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d4tj5\"" Jun 06 22:03:28.586393 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.586359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89a055b7-3e1a-4f23-85cf-e60006d08741-tmp-dir\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.586583 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.586414 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51877535-3dba-451c-912e-38b3b31cab69-cert\") pod \"ingress-canary-n5r5h\" (UID: \"51877535-3dba-451c-912e-38b3b31cab69\") " pod="openshift-ingress-canary/ingress-canary-n5r5h" Jun 06 22:03:28.586651 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.586611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9qp6\" (UniqueName: \"kubernetes.io/projected/51877535-3dba-451c-912e-38b3b31cab69-kube-api-access-b9qp6\") pod \"ingress-canary-n5r5h\" (UID: \"51877535-3dba-451c-912e-38b3b31cab69\") " pod="openshift-ingress-canary/ingress-canary-n5r5h" Jun 06 22:03:28.586706 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.586680 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.586759 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.586711 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/89a055b7-3e1a-4f23-85cf-e60006d08741-config-volume\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.586759 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.586740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mn4r\" (UniqueName: \"kubernetes.io/projected/89a055b7-3e1a-4f23-85cf-e60006d08741-kube-api-access-7mn4r\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.586887 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.586754 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89a055b7-3e1a-4f23-85cf-e60006d08741-tmp-dir\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.586887 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:28.586879 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:28.586980 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:28.586941 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls podName:89a055b7-3e1a-4f23-85cf-e60006d08741 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:29.08692028 +0000 UTC m=+36.085899768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls") pod "dns-default-ld6pc" (UID: "89a055b7-3e1a-4f23-85cf-e60006d08741") : secret "dns-default-metrics-tls" not found Jun 06 22:03:28.587440 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.587418 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/89a055b7-3e1a-4f23-85cf-e60006d08741-config-volume\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.591152 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.591015 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51877535-3dba-451c-912e-38b3b31cab69-cert\") pod \"ingress-canary-n5r5h\" (UID: \"51877535-3dba-451c-912e-38b3b31cab69\") " pod="openshift-ingress-canary/ingress-canary-n5r5h" Jun 06 22:03:28.596017 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.595944 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9qp6\" (UniqueName: \"kubernetes.io/projected/51877535-3dba-451c-912e-38b3b31cab69-kube-api-access-b9qp6\") pod \"ingress-canary-n5r5h\" (UID: \"51877535-3dba-451c-912e-38b3b31cab69\") " pod="openshift-ingress-canary/ingress-canary-n5r5h" Jun 06 22:03:28.596017 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.595944 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mn4r\" (UniqueName: \"kubernetes.io/projected/89a055b7-3e1a-4f23-85cf-e60006d08741-kube-api-access-7mn4r\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:28.745767 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:28.745728 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n5r5h" Jun 06 22:03:29.090858 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:29.090820 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:29.091033 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:29.090952 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:29.091033 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:29.091012 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls podName:89a055b7-3e1a-4f23-85cf-e60006d08741 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:30.090995294 +0000 UTC m=+37.089974759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls") pod "dns-default-ld6pc" (UID: "89a055b7-3e1a-4f23-85cf-e60006d08741") : secret "dns-default-metrics-tls" not found Jun 06 22:03:29.858554 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:29.858477 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n5r5h"] Jun 06 22:03:29.864197 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:03:29.864152 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51877535_3dba_451c_912e_38b3b31cab69.slice/crio-724b6188d388dd97d4859d62b4c4f9df7fb461ba4cff1797ce820c3815de748a WatchSource:0}: Error finding container 724b6188d388dd97d4859d62b4c4f9df7fb461ba4cff1797ce820c3815de748a: Status 404 returned error can't find the container with id 724b6188d388dd97d4859d62b4c4f9df7fb461ba4cff1797ce820c3815de748a Jun 06 22:03:30.099447 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:30.099346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:30.099592 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:30.099505 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:30.099592 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:30.099569 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls podName:89a055b7-3e1a-4f23-85cf-e60006d08741 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:32.09955479 +0000 UTC m=+39.098534255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls") pod "dns-default-ld6pc" (UID: "89a055b7-3e1a-4f23-85cf-e60006d08741") : secret "dns-default-metrics-tls" not found Jun 06 22:03:30.408736 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:30.408630 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:30.493523 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:30.493489 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:30.543235 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:30.543193 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:30.666912 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:30.666828 2580 generic.go:356] "Generic (PLEG): container finished" podID="22adddad-6c0c-4608-9c53-45b0644d677a" containerID="0aa2e80377c76a7df3fd5c91f8989e6ebd89315273048bbd81d47c11a162fcdd" exitCode=0 Jun 06 22:03:30.667066 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:30.666934 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" event={"ID":"22adddad-6c0c-4608-9c53-45b0644d677a","Type":"ContainerDied","Data":"0aa2e80377c76a7df3fd5c91f8989e6ebd89315273048bbd81d47c11a162fcdd"} Jun 06 22:03:30.668418 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:30.668393 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n5r5h" event={"ID":"51877535-3dba-451c-912e-38b3b31cab69","Type":"ContainerStarted","Data":"724b6188d388dd97d4859d62b4c4f9df7fb461ba4cff1797ce820c3815de748a"} Jun 06 22:03:31.618495 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:31.618454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:31.670434 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:31.670399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:31.673833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:31.673788 2580 generic.go:356] "Generic (PLEG): container finished" podID="22adddad-6c0c-4608-9c53-45b0644d677a" containerID="a60aee8adb81bd6c9fd9f1b6061fd7ef0f1eed397cee0b7a6e4283866efa9805" exitCode=0 Jun 06 22:03:31.673964 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:31.673861 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" event={"ID":"22adddad-6c0c-4608-9c53-45b0644d677a","Type":"ContainerDied","Data":"a60aee8adb81bd6c9fd9f1b6061fd7ef0f1eed397cee0b7a6e4283866efa9805"} Jun 06 22:03:31.675333 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:31.675305 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n5r5h" event={"ID":"51877535-3dba-451c-912e-38b3b31cab69","Type":"ContainerStarted","Data":"80110fb76839de930cbfecc90832d634ff6934e477b386073493ffae553a881f"} Jun 06 22:03:31.719383 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:31.719343 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n5r5h" podStartSLOduration=2.203735617 podStartE2EDuration="3.719328107s" totalImagesPullingTime="1.51559249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:31.718813173 +0000 UTC m=+38.717792649" watchObservedRunningTime="2026-06-06 22:03:31.719328107 +0000 UTC m=+38.718307588" Jun 06 22:03:32.115150 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:32.115107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:32.115297 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:32.115236 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:32.115297 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:32.115293 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls podName:89a055b7-3e1a-4f23-85cf-e60006d08741 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:36.115279009 +0000 UTC m=+43.114258471 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls") pod "dns-default-ld6pc" (UID: "89a055b7-3e1a-4f23-85cf-e60006d08741") : secret "dns-default-metrics-tls" not found Jun 06 22:03:32.680880 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:32.680838 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" event={"ID":"22adddad-6c0c-4608-9c53-45b0644d677a","Type":"ContainerStarted","Data":"8bd91851f78fd90d7c13a553ef25dc89f9188ec979435d2e3d4cd7744320dd46"} Jun 06 22:03:32.704799 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:32.704752 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hwmq5" podStartSLOduration=10.451742155 podStartE2EDuration="39.704739275s" totalImagesPullingTime="29.25299712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:03:32.703287703 +0000 UTC m=+39.702267186" watchObservedRunningTime="2026-06-06 22:03:32.704739275 +0000 UTC m=+39.703718759" Jun 06 22:03:32.824645 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:32.824617 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:32.887881 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:32.887849 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:33.239414 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:33.239356 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:33.292715 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:33.292679 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:36.140760 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:36.140725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:36.141128 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:36.140846 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:36.141128 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:36.140897 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls podName:89a055b7-3e1a-4f23-85cf-e60006d08741 nodeName:}" failed. No retries permitted until 2026-06-06 22:03:44.14088465 +0000 UTC m=+51.139864112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls") pod "dns-default-ld6pc" (UID: "89a055b7-3e1a-4f23-85cf-e60006d08741") : secret "dns-default-metrics-tls" not found Jun 06 22:03:36.990422 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:36.990387 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:37.058596 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:37.058563 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:40.062584 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:40.062547 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:44.193504 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:44.193465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:03:44.193925 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:44.193642 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:03:44.193925 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:44.193720 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls podName:89a055b7-3e1a-4f23-85cf-e60006d08741 nodeName:}" failed. No retries permitted until 2026-06-06 22:04:00.193704641 +0000 UTC m=+67.192684103 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls") pod "dns-default-ld6pc" (UID: "89a055b7-3e1a-4f23-85cf-e60006d08741") : secret "dns-default-metrics-tls" not found Jun 06 22:03:52.648548 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:52.648514 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74znx" Jun 06 22:03:52.796631 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:52.796598 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:52.854702 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:52.854664 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:53.582021 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:53.581971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:53.636768 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:53.636730 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:03:59.194126 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.194079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:03:59.196758 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.196739 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 06 22:03:59.205231 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:59.205212 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 06 22:03:59.205288 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:03:59.205276 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs podName:5f4eec8c-1779-4a2f-b0e0-a028da53560b nodeName:}" failed. No retries permitted until 2026-06-06 22:05:03.205254775 +0000 UTC m=+130.204234237 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs") pod "network-metrics-daemon-j7ksf" (UID: "5f4eec8c-1779-4a2f-b0e0-a028da53560b") : secret "metrics-daemon-secret" not found Jun 06 22:03:59.294473 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.294436 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:59.297403 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.297386 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 06 22:03:59.308924 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.308899 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db6d6e2f-7de5-4121-b6ca-d7f5aa168471-original-pull-secret\") pod \"global-pull-secret-syncer-m7gsp\" (UID: \"db6d6e2f-7de5-4121-b6ca-d7f5aa168471\") " pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:59.393700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.393671 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gsp" Jun 06 22:03:59.394900 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.394880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:59.398052 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.398034 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 06 22:03:59.408609 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.408587 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 06 22:03:59.418382 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.418358 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlsq6\" (UniqueName: \"kubernetes.io/projected/50ef0dd1-9d49-4216-8151-55e7a7fffd8b-kube-api-access-hlsq6\") pod \"network-check-target-vlk2c\" (UID: \"50ef0dd1-9d49-4216-8151-55e7a7fffd8b\") " pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:59.511685 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.511654 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-m7gsp"] Jun 06 22:03:59.515187 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:03:59.515151 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb6d6e2f_7de5_4121_b6ca_d7f5aa168471.slice/crio-639d03cb2253b497301aca5bebb056338f1373f1827d8f00ca9c57045ca234aa WatchSource:0}: Error finding container 639d03cb2253b497301aca5bebb056338f1373f1827d8f00ca9c57045ca234aa: Status 404 returned error can't find the container with id 639d03cb2253b497301aca5bebb056338f1373f1827d8f00ca9c57045ca234aa Jun 06 22:03:59.708433 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.708405 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ksb8v\"" Jun 06 22:03:59.716660 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.716638 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:03:59.743884 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.743858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-m7gsp" event={"ID":"db6d6e2f-7de5-4121-b6ca-d7f5aa168471","Type":"ContainerStarted","Data":"639d03cb2253b497301aca5bebb056338f1373f1827d8f00ca9c57045ca234aa"} Jun 06 22:03:59.834214 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:03:59.834183 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vlk2c"] Jun 06 22:03:59.838335 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:03:59.838305 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ef0dd1_9d49_4216_8151_55e7a7fffd8b.slice/crio-6e2fcae5920719c55455f83a17199ccfb96574cd1ac794e28633a25adf715564 WatchSource:0}: Error finding container 6e2fcae5920719c55455f83a17199ccfb96574cd1ac794e28633a25adf715564: Status 404 returned error can't find the container with id 6e2fcae5920719c55455f83a17199ccfb96574cd1ac794e28633a25adf715564 Jun 06 22:04:00.199996 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:00.199957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:04:00.200438 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:04:00.200128 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:04:00.200438 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:04:00.200214 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls podName:89a055b7-3e1a-4f23-85cf-e60006d08741 nodeName:}" failed. No retries permitted until 2026-06-06 22:04:32.200189945 +0000 UTC m=+99.199169408 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls") pod "dns-default-ld6pc" (UID: "89a055b7-3e1a-4f23-85cf-e60006d08741") : secret "dns-default-metrics-tls" not found Jun 06 22:04:00.748231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:00.748190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vlk2c" event={"ID":"50ef0dd1-9d49-4216-8151-55e7a7fffd8b","Type":"ContainerStarted","Data":"6e2fcae5920719c55455f83a17199ccfb96574cd1ac794e28633a25adf715564"} Jun 06 22:04:00.947175 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:00.947120 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:04:01.010214 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:01.010110 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:04:04.759178 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:04.759138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-m7gsp" event={"ID":"db6d6e2f-7de5-4121-b6ca-d7f5aa168471","Type":"ContainerStarted","Data":"03b25d55313da0b234f9a911a259e7fc15ef22ca01839d4732ff8970b04e3f74"} Jun 06 22:04:04.760661 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:04.760637 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vlk2c" event={"ID":"50ef0dd1-9d49-4216-8151-55e7a7fffd8b","Type":"ContainerStarted","Data":"fef2235522c90dbd69ffe1d5e0ca335f7eff1afcef98997994f8f3c702df3890"} Jun 06 22:04:04.760796 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:04.760775 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:04:04.774191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:04.774148 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-m7gsp" podStartSLOduration=66.87729528 podStartE2EDuration="1m11.774117995s" totalImagesPullingTime="4.896822715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:04:04.773840047 +0000 UTC m=+71.772819532" watchObservedRunningTime="2026-06-06 22:04:04.774117995 +0000 UTC m=+71.773097479" Jun 06 22:04:04.787388 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:04.787347 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vlk2c" podStartSLOduration=67.225069948 podStartE2EDuration="1m11.787333781s" totalImagesPullingTime="4.562263833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:04:04.786576784 +0000 UTC m=+71.785556268" watchObservedRunningTime="2026-06-06 22:04:04.787333781 +0000 UTC m=+71.786313265" Jun 06 22:04:07.857571 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:07.857536 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:04:07.975325 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:07.975288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:04:32.212285 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:32.212244 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:04:32.212712 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:04:32.212354 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:04:32.212712 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:04:32.212405 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls podName:89a055b7-3e1a-4f23-85cf-e60006d08741 nodeName:}" failed. No retries permitted until 2026-06-06 22:05:36.212392863 +0000 UTC m=+163.211372325 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls") pod "dns-default-ld6pc" (UID: "89a055b7-3e1a-4f23-85cf-e60006d08741") : secret "dns-default-metrics-tls" not found Jun 06 22:04:35.765952 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:04:35.765913 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vlk2c" Jun 06 22:05:03.211491 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:05:03.211448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:05:03.211991 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:05:03.211575 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 06 22:05:03.211991 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:05:03.211638 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs podName:5f4eec8c-1779-4a2f-b0e0-a028da53560b nodeName:}" failed. No retries permitted until 2026-06-06 22:07:05.211624423 +0000 UTC m=+252.210603884 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs") pod "network-metrics-daemon-j7ksf" (UID: "5f4eec8c-1779-4a2f-b0e0-a028da53560b") : secret "metrics-daemon-secret" not found Jun 06 22:05:31.416010 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:05:31.415955 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ld6pc" podUID="89a055b7-3e1a-4f23-85cf-e60006d08741" Jun 06 22:05:31.500374 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:05:31.500341 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-j7ksf" podUID="5f4eec8c-1779-4a2f-b0e0-a028da53560b" Jun 06 22:05:31.962465 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:05:31.962435 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld6pc" Jun 06 22:05:36.237054 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:05:36.237015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:05:36.237461 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:05:36.237147 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 06 22:05:36.237461 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:05:36.237211 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls podName:89a055b7-3e1a-4f23-85cf-e60006d08741 nodeName:}" failed. No retries permitted until 2026-06-06 22:07:38.237196345 +0000 UTC m=+285.236175806 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls") pod "dns-default-ld6pc" (UID: "89a055b7-3e1a-4f23-85cf-e60006d08741") : secret "dns-default-metrics-tls" not found Jun 06 22:05:46.483312 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:05:46.483211 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:05:53.922670 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:05:53.922639 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:13.955292 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:13.955256 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q"] Jun 06 22:06:13.957982 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:13.957964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:13.960782 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:13.960765 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 06 22:06:13.961120 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:13.961091 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 06 22:06:13.961762 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:13.961746 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z59hn\"" Jun 06 22:06:13.977672 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:13.977646 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q"] Jun 06 22:06:14.048733 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.048698 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f"] Jun 06 22:06:14.051462 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.051423 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v"] Jun 06 22:06:14.051573 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.051536 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f" Jun 06 22:06:14.054025 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.054005 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc"] Jun 06 22:06:14.054144 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.054124 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.054188 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.054145 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8xvr4\"" Jun 06 22:06:14.054668 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.054648 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:06:14.054668 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.054660 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 06 22:06:14.056638 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.056620 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 06 22:06:14.056750 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.056717 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 06 22:06:14.056963 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.056945 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-4ngmg"] Jun 06 22:06:14.057091 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.056992 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jk75p\"" Jun 06 22:06:14.057091 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.057043 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:06:14.057230 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.057115 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:14.057568 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.057547 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 06 22:06:14.059619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.059594 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:06:14.059722 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.059596 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 06 22:06:14.060164 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.060023 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2sccd\"" Jun 06 22:06:14.060279 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.060213 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 06 22:06:14.060555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.060537 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-78476cbbd5-ldsl9"] Jun 06 22:06:14.062647 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.062627 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.064365 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.064315 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.066007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.065988 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 06 22:06:14.066255 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.066096 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 06 22:06:14.066441 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.066422 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-cf9c6\"" Jun 06 22:06:14.067961 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.067542 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 06 22:06:14.067961 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.067643 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:06:14.067961 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.067548 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 06 22:06:14.068484 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.068072 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 06 22:06:14.068484 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.068253 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v"] Jun 06 22:06:14.068932 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.068910 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 06 22:06:14.069525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.069172 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 06 22:06:14.069525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.069250 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 06 22:06:14.069525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.069365 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d7s7t\"" Jun 06 22:06:14.069525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.069476 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 06 22:06:14.070743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.070722 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f"] Jun 06 22:06:14.071792 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.071770 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-4ngmg"] Jun 06 22:06:14.072261 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.072242 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 06 22:06:14.079582 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.079562 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc"] Jun 06 22:06:14.080477 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.080456 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78476cbbd5-ldsl9"] Jun 06 22:06:14.084548 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.084526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff32d99a-8480-4abf-91d3-8eb414d4bf85-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:14.084651 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.084578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:14.184244 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.184212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:14.185202 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-stats-auth\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.185265 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0444ca20-8bd0-4be6-a4ff-a4afef1f020e-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-vsd4v\" (UID: \"0444ca20-8bd0-4be6-a4ff-a4afef1f020e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.185265 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwppk\" (UniqueName: \"kubernetes.io/projected/d78b6164-d3a6-4217-a627-f2e0fb3ade06-kube-api-access-mwppk\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.185365 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc8l6\" (UniqueName: \"kubernetes.io/projected/e7a21203-dae9-4818-a0cb-8115e6c92d19-kube-api-access-xc8l6\") pod \"volume-data-source-validator-78dd696465-r9d9f\" (UID: \"e7a21203-dae9-4818-a0cb-8115e6c92d19\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f" Jun 06 22:06:14.185365 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185352 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.185437 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185382 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zmzp\" (UniqueName: \"kubernetes.io/projected/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-kube-api-access-9zmzp\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.185437 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.185437 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:14.185547 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185445 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d78b6164-d3a6-4217-a627-f2e0fb3ade06-config\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.185547 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185499 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff32d99a-8480-4abf-91d3-8eb414d4bf85-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:14.185547 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d78b6164-d3a6-4217-a627-f2e0fb3ade06-serving-cert\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.185547 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0444ca20-8bd0-4be6-a4ff-a4afef1f020e-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-vsd4v\" (UID: \"0444ca20-8bd0-4be6-a4ff-a4afef1f020e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.185700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2g4s\" (UniqueName: \"kubernetes.io/projected/0444ca20-8bd0-4be6-a4ff-a4afef1f020e-kube-api-access-h2g4s\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-vsd4v\" (UID: \"0444ca20-8bd0-4be6-a4ff-a4afef1f020e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.185700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185577 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d78b6164-d3a6-4217-a627-f2e0fb3ade06-trusted-ca\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.185700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:14.185700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185646 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-default-certificate\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.185700 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.185671 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 06 22:06:14.185700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.185671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7p4h\" (UniqueName: \"kubernetes.io/projected/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-kube-api-access-p7p4h\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:14.185964 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.185732 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert podName:ff32d99a-8480-4abf-91d3-8eb414d4bf85 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:14.685710728 +0000 UTC m=+201.684690194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-hhv9q" (UID: "ff32d99a-8480-4abf-91d3-8eb414d4bf85") : secret "networking-console-plugin-cert" not found Jun 06 22:06:14.186114 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.186097 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff32d99a-8480-4abf-91d3-8eb414d4bf85-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:14.233724 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.233661 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:14.286546 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-stats-auth\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.286640 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286560 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0444ca20-8bd0-4be6-a4ff-a4afef1f020e-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-vsd4v\" (UID: \"0444ca20-8bd0-4be6-a4ff-a4afef1f020e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.286640 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwppk\" (UniqueName: \"kubernetes.io/projected/d78b6164-d3a6-4217-a627-f2e0fb3ade06-kube-api-access-mwppk\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.286640 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc8l6\" (UniqueName: \"kubernetes.io/projected/e7a21203-dae9-4818-a0cb-8115e6c92d19-kube-api-access-xc8l6\") pod \"volume-data-source-validator-78dd696465-r9d9f\" (UID: \"e7a21203-dae9-4818-a0cb-8115e6c92d19\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f" Jun 06 22:06:14.286640 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.286868 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286647 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zmzp\" (UniqueName: \"kubernetes.io/projected/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-kube-api-access-9zmzp\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.286868 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.286868 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286683 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:14.286868 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d78b6164-d3a6-4217-a627-f2e0fb3ade06-config\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.286868 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286731 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d78b6164-d3a6-4217-a627-f2e0fb3ade06-serving-cert\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.286868 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0444ca20-8bd0-4be6-a4ff-a4afef1f020e-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-vsd4v\" (UID: \"0444ca20-8bd0-4be6-a4ff-a4afef1f020e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.286868 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2g4s\" (UniqueName: \"kubernetes.io/projected/0444ca20-8bd0-4be6-a4ff-a4afef1f020e-kube-api-access-h2g4s\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-vsd4v\" (UID: \"0444ca20-8bd0-4be6-a4ff-a4afef1f020e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.286868 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286842 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d78b6164-d3a6-4217-a627-f2e0fb3ade06-trusted-ca\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.287328 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-default-certificate\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.287328 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.286906 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:14.786882039 +0000 UTC m=+201.785861518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : configmap references non-existent config key: service-ca.crt Jun 06 22:06:14.287328 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.286937 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 06 22:06:14.287328 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.286951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7p4h\" (UniqueName: \"kubernetes.io/projected/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-kube-api-access-p7p4h\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:14.287328 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.286995 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:14.786980622 +0000 UTC m=+201.785960091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : secret "router-metrics-certs-default" not found Jun 06 22:06:14.287328 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.287123 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 06 22:06:14.287328 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.287164 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls podName:0d2da874-3f8b-4ef2-adb9-f9d1d04680ac nodeName:}" failed. No retries permitted until 2026-06-06 22:06:14.787151651 +0000 UTC m=+201.786131113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-s8wpc" (UID: "0d2da874-3f8b-4ef2-adb9-f9d1d04680ac") : secret "samples-operator-tls" not found Jun 06 22:06:14.287735 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.287507 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d78b6164-d3a6-4217-a627-f2e0fb3ade06-config\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.287735 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.287636 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0444ca20-8bd0-4be6-a4ff-a4afef1f020e-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-vsd4v\" (UID: \"0444ca20-8bd0-4be6-a4ff-a4afef1f020e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.288167 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.288143 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d78b6164-d3a6-4217-a627-f2e0fb3ade06-trusted-ca\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.289250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.289222 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-stats-auth\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.289352 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.289272 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-default-certificate\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.289399 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.289353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0444ca20-8bd0-4be6-a4ff-a4afef1f020e-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-vsd4v\" (UID: \"0444ca20-8bd0-4be6-a4ff-a4afef1f020e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.289566 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.289549 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d78b6164-d3a6-4217-a627-f2e0fb3ade06-serving-cert\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.298033 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.298008 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2g4s\" (UniqueName: \"kubernetes.io/projected/0444ca20-8bd0-4be6-a4ff-a4afef1f020e-kube-api-access-h2g4s\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-vsd4v\" (UID: \"0444ca20-8bd0-4be6-a4ff-a4afef1f020e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.298152 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.298089 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc8l6\" (UniqueName: \"kubernetes.io/projected/e7a21203-dae9-4818-a0cb-8115e6c92d19-kube-api-access-xc8l6\") pod \"volume-data-source-validator-78dd696465-r9d9f\" (UID: \"e7a21203-dae9-4818-a0cb-8115e6c92d19\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f" Jun 06 22:06:14.298152 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.298112 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwppk\" (UniqueName: \"kubernetes.io/projected/d78b6164-d3a6-4217-a627-f2e0fb3ade06-kube-api-access-mwppk\") pod \"console-operator-668fdbf85c-4ngmg\" (UID: \"d78b6164-d3a6-4217-a627-f2e0fb3ade06\") " pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.298268 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.298190 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zmzp\" (UniqueName: \"kubernetes.io/projected/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-kube-api-access-9zmzp\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.298424 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.298406 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7p4h\" (UniqueName: \"kubernetes.io/projected/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-kube-api-access-p7p4h\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:14.363994 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.363960 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f" Jun 06 22:06:14.372892 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.372873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" Jun 06 22:06:14.386348 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.386321 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:14.503443 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.503368 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f"] Jun 06 22:06:14.507514 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:14.507480 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a21203_dae9_4818_a0cb_8115e6c92d19.slice/crio-48e46549b7a5ce69629d7323dfc0d6192d46eb384ee3f9e43c68df581ed6ccd6 WatchSource:0}: Error finding container 48e46549b7a5ce69629d7323dfc0d6192d46eb384ee3f9e43c68df581ed6ccd6: Status 404 returned error can't find the container with id 48e46549b7a5ce69629d7323dfc0d6192d46eb384ee3f9e43c68df581ed6ccd6 Jun 06 22:06:14.520043 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.520015 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v"] Jun 06 22:06:14.522928 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:14.522901 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0444ca20_8bd0_4be6_a4ff_a4afef1f020e.slice/crio-7036b4860fa05fc10eb6d96b652bd9d4624fcda900b85e600868a87078f98e82 WatchSource:0}: Error finding container 7036b4860fa05fc10eb6d96b652bd9d4624fcda900b85e600868a87078f98e82: Status 404 returned error can't find the container with id 7036b4860fa05fc10eb6d96b652bd9d4624fcda900b85e600868a87078f98e82 Jun 06 22:06:14.538832 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.538796 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-4ngmg"] Jun 06 22:06:14.541772 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:14.541749 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78b6164_d3a6_4217_a627_f2e0fb3ade06.slice/crio-b248425afe0dfad1ccab10f31c17cb655623517176a12510caed2b1c423db3b6 WatchSource:0}: Error finding container b248425afe0dfad1ccab10f31c17cb655623517176a12510caed2b1c423db3b6: Status 404 returned error can't find the container with id b248425afe0dfad1ccab10f31c17cb655623517176a12510caed2b1c423db3b6 Jun 06 22:06:14.690825 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.690769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:14.691027 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.690957 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 06 22:06:14.691099 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.691037 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert podName:ff32d99a-8480-4abf-91d3-8eb414d4bf85 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:15.691015907 +0000 UTC m=+202.689995386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-hhv9q" (UID: "ff32d99a-8480-4abf-91d3-8eb414d4bf85") : secret "networking-console-plugin-cert" not found Jun 06 22:06:14.792211 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.792119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.792211 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.792171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:14.792211 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:14.792197 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:14.792444 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.792286 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 06 22:06:14.792444 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.792342 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:15.792327043 +0000 UTC m=+202.791306505 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : secret "router-metrics-certs-default" not found Jun 06 22:06:14.792444 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.792292 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 06 22:06:14.792444 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.792415 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:15.792399651 +0000 UTC m=+202.791379121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : configmap references non-existent config key: service-ca.crt Jun 06 22:06:14.792444 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:14.792435 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls podName:0d2da874-3f8b-4ef2-adb9-f9d1d04680ac nodeName:}" failed. No retries permitted until 2026-06-06 22:06:15.792427895 +0000 UTC m=+202.791407356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-s8wpc" (UID: "0d2da874-3f8b-4ef2-adb9-f9d1d04680ac") : secret "samples-operator-tls" not found Jun 06 22:06:15.063260 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:15.063169 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" event={"ID":"d78b6164-d3a6-4217-a627-f2e0fb3ade06","Type":"ContainerStarted","Data":"b248425afe0dfad1ccab10f31c17cb655623517176a12510caed2b1c423db3b6"} Jun 06 22:06:15.064504 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:15.064475 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" event={"ID":"0444ca20-8bd0-4be6-a4ff-a4afef1f020e","Type":"ContainerStarted","Data":"7036b4860fa05fc10eb6d96b652bd9d4624fcda900b85e600868a87078f98e82"} Jun 06 22:06:15.065793 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:15.065763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f" event={"ID":"e7a21203-dae9-4818-a0cb-8115e6c92d19","Type":"ContainerStarted","Data":"48e46549b7a5ce69629d7323dfc0d6192d46eb384ee3f9e43c68df581ed6ccd6"} Jun 06 22:06:15.701002 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:15.700962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:15.701254 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:15.701230 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 06 22:06:15.701370 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:15.701320 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert podName:ff32d99a-8480-4abf-91d3-8eb414d4bf85 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:17.701299355 +0000 UTC m=+204.700278821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-hhv9q" (UID: "ff32d99a-8480-4abf-91d3-8eb414d4bf85") : secret "networking-console-plugin-cert" not found Jun 06 22:06:15.801851 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:15.801791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:15.802037 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:15.801868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:15.802037 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:15.801899 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:15.802037 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:15.801948 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:17.801924689 +0000 UTC m=+204.800904152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : configmap references non-existent config key: service-ca.crt Jun 06 22:06:15.802037 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:15.802013 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 06 22:06:15.802256 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:15.802063 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls podName:0d2da874-3f8b-4ef2-adb9-f9d1d04680ac nodeName:}" failed. No retries permitted until 2026-06-06 22:06:17.802048086 +0000 UTC m=+204.801027562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-s8wpc" (UID: "0d2da874-3f8b-4ef2-adb9-f9d1d04680ac") : secret "samples-operator-tls" not found Jun 06 22:06:15.802256 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:15.802118 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 06 22:06:15.802256 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:15.802149 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:17.802138319 +0000 UTC m=+204.801117786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : secret "router-metrics-certs-default" not found Jun 06 22:06:16.070363 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:16.070324 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f" event={"ID":"e7a21203-dae9-4818-a0cb-8115e6c92d19","Type":"ContainerStarted","Data":"07e3bf1f646801aeef85df2e94702422ffd58c87865687df5ddc91d773432daf"} Jun 06 22:06:16.089078 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:16.089006 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-r9d9f" podStartSLOduration=0.601341464 podStartE2EDuration="2.088989317s" totalImagesPullingTime="1.487647853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:16.084295762 +0000 UTC m=+203.083275247" watchObservedRunningTime="2026-06-06 22:06:16.088989317 +0000 UTC m=+203.087968804" Jun 06 22:06:16.153684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:16.153653 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:16.199833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:16.199783 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:17.719473 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:17.719439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:17.719963 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:17.719600 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 06 22:06:17.719963 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:17.719677 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert podName:ff32d99a-8480-4abf-91d3-8eb414d4bf85 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:21.719656258 +0000 UTC m=+208.718635721 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-hhv9q" (UID: "ff32d99a-8480-4abf-91d3-8eb414d4bf85") : secret "networking-console-plugin-cert" not found Jun 06 22:06:17.820220 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:17.820190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:17.820220 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:17.820220 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:17.820367 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:17.820324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:17.820367 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:17.820339 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 06 22:06:17.820467 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:17.820399 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:21.820379267 +0000 UTC m=+208.819358742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : secret "router-metrics-certs-default" not found Jun 06 22:06:17.820467 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:17.820402 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 06 22:06:17.820467 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:17.820416 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:21.820405363 +0000 UTC m=+208.819384825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : configmap references non-existent config key: service-ca.crt Jun 06 22:06:17.820467 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:17.820454 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls podName:0d2da874-3f8b-4ef2-adb9-f9d1d04680ac nodeName:}" failed. No retries permitted until 2026-06-06 22:06:21.820437814 +0000 UTC m=+208.819417279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-s8wpc" (UID: "0d2da874-3f8b-4ef2-adb9-f9d1d04680ac") : secret "samples-operator-tls" not found Jun 06 22:06:17.957487 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:17.957453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:18.056490 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:18.056418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:18.078292 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:18.078268 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/0.log" Jun 06 22:06:18.078412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:18.078301 2580 generic.go:356] "Generic (PLEG): container finished" podID="d78b6164-d3a6-4217-a627-f2e0fb3ade06" containerID="ace7e5be9b65fa8a303f9b0e23b7112bfbccf9eed1a219119b0f9dff34885ef9" exitCode=255 Jun 06 22:06:18.078412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:18.078330 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" event={"ID":"d78b6164-d3a6-4217-a627-f2e0fb3ade06","Type":"ContainerDied","Data":"ace7e5be9b65fa8a303f9b0e23b7112bfbccf9eed1a219119b0f9dff34885ef9"} Jun 06 22:06:18.078602 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:18.078587 2580 scope.go:117] "RemoveContainer" containerID="ace7e5be9b65fa8a303f9b0e23b7112bfbccf9eed1a219119b0f9dff34885ef9" Jun 06 22:06:18.079837 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:18.079795 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" event={"ID":"0444ca20-8bd0-4be6-a4ff-a4afef1f020e","Type":"ContainerStarted","Data":"3ed95d2a522f083a399b939e97530efdb00e1b511f6b602786c1e3878f6bef18"} Jun 06 22:06:18.107791 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:18.107753 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" podStartSLOduration=1.59910117 podStartE2EDuration="4.107738439s" totalImagesPullingTime="2.508637269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:18.107501605 +0000 UTC m=+205.106481089" watchObservedRunningTime="2026-06-06 22:06:18.107738439 +0000 UTC m=+205.106717924" Jun 06 22:06:18.362984 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:18.362909 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:18.462294 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:18.462252 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:19.057745 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.057705 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:19.085285 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.085252 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/1.log" Jun 06 22:06:19.085712 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.085696 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/0.log" Jun 06 22:06:19.085865 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.085731 2580 generic.go:356] "Generic (PLEG): container finished" podID="d78b6164-d3a6-4217-a627-f2e0fb3ade06" containerID="6525dc8d114e028aea90734001483e2053dd3c3691601a678eb679850adf7ba3" exitCode=255 Jun 06 22:06:19.085865 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.085766 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" event={"ID":"d78b6164-d3a6-4217-a627-f2e0fb3ade06","Type":"ContainerDied","Data":"6525dc8d114e028aea90734001483e2053dd3c3691601a678eb679850adf7ba3"} Jun 06 22:06:19.085968 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.085868 2580 scope.go:117] "RemoveContainer" containerID="ace7e5be9b65fa8a303f9b0e23b7112bfbccf9eed1a219119b0f9dff34885ef9" Jun 06 22:06:19.086125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.086105 2580 scope.go:117] "RemoveContainer" containerID="6525dc8d114e028aea90734001483e2053dd3c3691601a678eb679850adf7ba3" Jun 06 22:06:19.086334 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:19.086309 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-4ngmg_openshift-console-operator(d78b6164-d3a6-4217-a627-f2e0fb3ade06)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" podUID="d78b6164-d3a6-4217-a627-f2e0fb3ade06" Jun 06 22:06:19.153346 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.153302 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:19.251136 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.251099 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:19.877381 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:19.877354 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bp8js_c6c24d13-14a9-4f94-8112-0ff9d1f62946/dns-node-resolver/0.log" Jun 06 22:06:20.090307 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.090282 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/1.log" Jun 06 22:06:20.090687 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.090591 2580 scope.go:117] "RemoveContainer" containerID="6525dc8d114e028aea90734001483e2053dd3c3691601a678eb679850adf7ba3" Jun 06 22:06:20.090763 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:20.090741 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-4ngmg_openshift-console-operator(d78b6164-d3a6-4217-a627-f2e0fb3ade06)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" podUID="d78b6164-d3a6-4217-a627-f2e0fb3ade06" Jun 06 22:06:20.612820 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.612775 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-g5rzd"] Jun 06 22:06:20.616889 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.616872 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.619692 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.619667 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 06 22:06:20.620870 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.620849 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 06 22:06:20.620985 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.620856 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-28tff\"" Jun 06 22:06:20.620985 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.620918 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 06 22:06:20.620985 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.620889 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 06 22:06:20.624563 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.624540 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-g5rzd"] Jun 06 22:06:20.647698 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.647671 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:20.704220 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.704191 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:20.744851 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.744831 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gd88\" (UniqueName: \"kubernetes.io/projected/d0c45f99-d2a0-4524-9919-2cd5a600461b-kube-api-access-8gd88\") pod \"service-ca-7d74d578b8-g5rzd\" (UID: \"d0c45f99-d2a0-4524-9919-2cd5a600461b\") " pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.744974 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.744956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d0c45f99-d2a0-4524-9919-2cd5a600461b-signing-cabundle\") pod \"service-ca-7d74d578b8-g5rzd\" (UID: \"d0c45f99-d2a0-4524-9919-2cd5a600461b\") " pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.745021 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.745000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d0c45f99-d2a0-4524-9919-2cd5a600461b-signing-key\") pod \"service-ca-7d74d578b8-g5rzd\" (UID: \"d0c45f99-d2a0-4524-9919-2cd5a600461b\") " pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.846361 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.846318 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gd88\" (UniqueName: \"kubernetes.io/projected/d0c45f99-d2a0-4524-9919-2cd5a600461b-kube-api-access-8gd88\") pod \"service-ca-7d74d578b8-g5rzd\" (UID: \"d0c45f99-d2a0-4524-9919-2cd5a600461b\") " pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.846501 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.846491 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d0c45f99-d2a0-4524-9919-2cd5a600461b-signing-cabundle\") pod \"service-ca-7d74d578b8-g5rzd\" (UID: \"d0c45f99-d2a0-4524-9919-2cd5a600461b\") " pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.846543 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.846512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d0c45f99-d2a0-4524-9919-2cd5a600461b-signing-key\") pod \"service-ca-7d74d578b8-g5rzd\" (UID: \"d0c45f99-d2a0-4524-9919-2cd5a600461b\") " pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.847166 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.847142 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d0c45f99-d2a0-4524-9919-2cd5a600461b-signing-cabundle\") pod \"service-ca-7d74d578b8-g5rzd\" (UID: \"d0c45f99-d2a0-4524-9919-2cd5a600461b\") " pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.848739 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.848716 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d0c45f99-d2a0-4524-9919-2cd5a600461b-signing-key\") pod \"service-ca-7d74d578b8-g5rzd\" (UID: \"d0c45f99-d2a0-4524-9919-2cd5a600461b\") " pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.854389 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.854365 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gd88\" (UniqueName: \"kubernetes.io/projected/d0c45f99-d2a0-4524-9919-2cd5a600461b-kube-api-access-8gd88\") pod \"service-ca-7d74d578b8-g5rzd\" (UID: \"d0c45f99-d2a0-4524-9919-2cd5a600461b\") " pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:20.876989 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.876939 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-txkmb_9862602e-db24-4b4a-ab24-a6b16129404e/node-ca/0.log" Jun 06 22:06:20.926252 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:20.926230 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" Jun 06 22:06:21.046469 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:21.046440 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-g5rzd"] Jun 06 22:06:21.049190 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:21.049157 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c45f99_d2a0_4524_9919_2cd5a600461b.slice/crio-5bd97be457a27c859fd32fd65e902204f8111378023cdf597a6b6df197c59048 WatchSource:0}: Error finding container 5bd97be457a27c859fd32fd65e902204f8111378023cdf597a6b6df197c59048: Status 404 returned error can't find the container with id 5bd97be457a27c859fd32fd65e902204f8111378023cdf597a6b6df197c59048 Jun 06 22:06:21.094501 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:21.094471 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" event={"ID":"d0c45f99-d2a0-4524-9919-2cd5a600461b","Type":"ContainerStarted","Data":"5bd97be457a27c859fd32fd65e902204f8111378023cdf597a6b6df197c59048"} Jun 06 22:06:21.513692 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:21.513652 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:21.677774 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:21.677745 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n5r5h_51877535-3dba-451c-912e-38b3b31cab69/serve-healthcheck-canary/0.log" Jun 06 22:06:21.754055 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:21.754016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:21.754246 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:21.754193 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 06 22:06:21.754313 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:21.754269 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert podName:ff32d99a-8480-4abf-91d3-8eb414d4bf85 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:29.754248279 +0000 UTC m=+216.753227743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-hhv9q" (UID: "ff32d99a-8480-4abf-91d3-8eb414d4bf85") : secret "networking-console-plugin-cert" not found Jun 06 22:06:21.854943 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:21.854861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:21.854943 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:21.854907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:21.854943 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:21.854935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:21.855364 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:21.855036 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 06 22:06:21.855364 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:21.855072 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 06 22:06:21.855364 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:21.855039 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:29.855021666 +0000 UTC m=+216.854001143 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : configmap references non-existent config key: service-ca.crt Jun 06 22:06:21.855364 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:21.855114 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:29.855092884 +0000 UTC m=+216.854072345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : secret "router-metrics-certs-default" not found Jun 06 22:06:21.855364 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:21.855166 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls podName:0d2da874-3f8b-4ef2-adb9-f9d1d04680ac nodeName:}" failed. No retries permitted until 2026-06-06 22:06:29.855128322 +0000 UTC m=+216.854107803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-s8wpc" (UID: "0d2da874-3f8b-4ef2-adb9-f9d1d04680ac") : secret "samples-operator-tls" not found Jun 06 22:06:23.102928 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:23.102890 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" event={"ID":"d0c45f99-d2a0-4524-9919-2cd5a600461b","Type":"ContainerStarted","Data":"d3b50a9585ee23dc3b3612beaedff9f1ed03137b35bfb6946b95f17709dc4c4d"} Jun 06 22:06:23.120444 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:23.120396 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-g5rzd" podStartSLOduration=1.600922277 podStartE2EDuration="3.12038093s" totalImagesPullingTime="1.519458653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:23.119909418 +0000 UTC m=+210.118888895" watchObservedRunningTime="2026-06-06 22:06:23.12038093 +0000 UTC m=+210.119360414" Jun 06 22:06:24.387029 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:24.386992 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:24.387430 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:24.387046 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:24.387430 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:24.387382 2580 scope.go:117] "RemoveContainer" containerID="6525dc8d114e028aea90734001483e2053dd3c3691601a678eb679850adf7ba3" Jun 06 22:06:24.387549 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:24.387529 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-4ngmg_openshift-console-operator(d78b6164-d3a6-4217-a627-f2e0fb3ade06)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" podUID="d78b6164-d3a6-4217-a627-f2e0fb3ade06" Jun 06 22:06:24.533584 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:24.533519 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:24.609991 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:24.609953 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:24.711319 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:24.711290 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:24.812435 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:24.812394 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:24.881479 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:24.881440 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:25.148302 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:25.148269 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:26.158569 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:26.158533 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:26.540272 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:26.540186 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:26.743698 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:26.743666 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:26.943526 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:26.943496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:27.157870 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:27.157838 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:27.868156 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:27.868127 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:28.143800 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:28.143770 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:28.244195 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:28.244169 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:28.383986 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:28.383955 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:28.545392 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:28.545309 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:28.761592 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:28.761563 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:28.946317 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:28.946287 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:29.352495 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:29.352419 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:29.811223 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:29.811189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:29.811390 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:29.811332 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 06 22:06:29.811432 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:29.811403 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert podName:ff32d99a-8480-4abf-91d3-8eb414d4bf85 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:45.811387586 +0000 UTC m=+232.810367048 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-hhv9q" (UID: "ff32d99a-8480-4abf-91d3-8eb414d4bf85") : secret "networking-console-plugin-cert" not found Jun 06 22:06:29.911676 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:29.911648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:29.911676 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:29.911681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:29.911901 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:29.911698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:29.911901 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:29.911881 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle podName:17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d nodeName:}" failed. No retries permitted until 2026-06-06 22:06:45.911857838 +0000 UTC m=+232.910837304 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle") pod "router-default-78476cbbd5-ldsl9" (UID: "17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d") : configmap references non-existent config key: service-ca.crt Jun 06 22:06:29.914127 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:29.914099 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d2da874-3f8b-4ef2-adb9-f9d1d04680ac-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-s8wpc\" (UID: \"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:29.914239 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:29.914130 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-metrics-certs\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:29.945549 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:29.945519 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:29.979645 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:29.979623 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" Jun 06 22:06:30.104276 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:30.104251 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc"] Jun 06 22:06:30.543341 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:30.543308 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:30.744659 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:30.744613 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:30.944665 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:30.944631 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:31.042345 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:31.042310 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:31.129719 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:31.129637 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" event={"ID":"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac","Type":"ContainerStarted","Data":"ad5c8088b9c8b8b9c6a5f90beca562858c13455f4d13ac64884776cc79f09671"} Jun 06 22:06:31.142339 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:31.142305 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:31.243287 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:31.243191 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:32.144968 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:32.144936 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:32.417644 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:32.417606 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:32.741279 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:32.741202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:32.945685 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:32.945654 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:33.137858 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:33.137760 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" event={"ID":"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac","Type":"ContainerStarted","Data":"0d4f847d218ee7b14044faaa4b348a1afbc73b6bdd5092e47f0cb885b2d7032d"} Jun 06 22:06:33.137858 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:33.137833 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" event={"ID":"0d2da874-3f8b-4ef2-adb9-f9d1d04680ac","Type":"ContainerStarted","Data":"798690d57f6d6adf7a95be880936dfcb744754033debde2dc89c3589bd8d831b"} Jun 06 22:06:33.155691 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:33.155649 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-s8wpc" podStartSLOduration=17.151753534 podStartE2EDuration="19.155636112s" totalImagesPullingTime="2.003882578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:33.154877436 +0000 UTC m=+220.153856929" watchObservedRunningTime="2026-06-06 22:06:33.155636112 +0000 UTC m=+220.154615596" Jun 06 22:06:33.536117 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:33.536080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:33.942048 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:33.942017 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:34.341756 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:34.341666 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:34.541735 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:34.541694 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:34.950588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:34.950549 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:35.151221 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:35.151177 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:39.483160 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.483131 2580 scope.go:117] "RemoveContainer" containerID="6525dc8d114e028aea90734001483e2053dd3c3691601a678eb679850adf7ba3" Jun 06 22:06:39.769818 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.769715 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fzq8w"] Jun 06 22:06:39.773328 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.773306 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.778815 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.778782 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 06 22:06:39.780067 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.780047 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 06 22:06:39.780168 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.780088 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vjws4\"" Jun 06 22:06:39.780551 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.780537 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 06 22:06:39.781077 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.781060 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 06 22:06:39.787111 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.787091 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fzq8w"] Jun 06 22:06:39.883610 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.883575 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56f37b5d-b920-4c7c-af0e-d35708ff9d95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.883774 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.883630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56f37b5d-b920-4c7c-af0e-d35708ff9d95-crio-socket\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.883774 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.883767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56f37b5d-b920-4c7c-af0e-d35708ff9d95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.883927 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.883838 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56f37b5d-b920-4c7c-af0e-d35708ff9d95-data-volume\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.883927 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.883865 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pprhz\" (UniqueName: \"kubernetes.io/projected/56f37b5d-b920-4c7c-af0e-d35708ff9d95-kube-api-access-pprhz\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.924121 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.924085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:39.984339 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.984313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56f37b5d-b920-4c7c-af0e-d35708ff9d95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.984483 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.984351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56f37b5d-b920-4c7c-af0e-d35708ff9d95-data-volume\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.984483 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.984369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pprhz\" (UniqueName: \"kubernetes.io/projected/56f37b5d-b920-4c7c-af0e-d35708ff9d95-kube-api-access-pprhz\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.984483 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.984390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56f37b5d-b920-4c7c-af0e-d35708ff9d95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.984483 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.984418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56f37b5d-b920-4c7c-af0e-d35708ff9d95-crio-socket\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.984483 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.984479 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56f37b5d-b920-4c7c-af0e-d35708ff9d95-crio-socket\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.984733 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.984711 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56f37b5d-b920-4c7c-af0e-d35708ff9d95-data-volume\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.985022 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.984998 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56f37b5d-b920-4c7c-af0e-d35708ff9d95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.986598 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.986581 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56f37b5d-b920-4c7c-af0e-d35708ff9d95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:39.995724 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:39.995703 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pprhz\" (UniqueName: \"kubernetes.io/projected/56f37b5d-b920-4c7c-af0e-d35708ff9d95-kube-api-access-pprhz\") pod \"insights-runtime-extractor-fzq8w\" (UID: \"56f37b5d-b920-4c7c-af0e-d35708ff9d95\") " pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:40.081560 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:40.081500 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fzq8w" Jun 06 22:06:40.161320 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:40.161294 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:06:40.161776 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:40.161760 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/1.log" Jun 06 22:06:40.161846 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:40.161797 2580 generic.go:356] "Generic (PLEG): container finished" podID="d78b6164-d3a6-4217-a627-f2e0fb3ade06" containerID="71fbf2a2ae221061baf87088d62cf9124eeb6d40a913cbe71544dff62302375d" exitCode=255 Jun 06 22:06:40.161901 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:40.161850 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" event={"ID":"d78b6164-d3a6-4217-a627-f2e0fb3ade06","Type":"ContainerDied","Data":"71fbf2a2ae221061baf87088d62cf9124eeb6d40a913cbe71544dff62302375d"} Jun 06 22:06:40.162224 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:40.161909 2580 scope.go:117] "RemoveContainer" containerID="6525dc8d114e028aea90734001483e2053dd3c3691601a678eb679850adf7ba3" Jun 06 22:06:40.162540 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:40.162338 2580 scope.go:117] "RemoveContainer" containerID="71fbf2a2ae221061baf87088d62cf9124eeb6d40a913cbe71544dff62302375d" Jun 06 22:06:40.162702 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:40.162534 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-4ngmg_openshift-console-operator(d78b6164-d3a6-4217-a627-f2e0fb3ade06)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" podUID="d78b6164-d3a6-4217-a627-f2e0fb3ade06" Jun 06 22:06:40.212971 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:40.212945 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fzq8w"] Jun 06 22:06:40.215869 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:40.215844 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56f37b5d_b920_4c7c_af0e_d35708ff9d95.slice/crio-cc33593250837cf3a58098b55f85baf8698535d3c3c29488deeb89cc8349eed4 WatchSource:0}: Error finding container cc33593250837cf3a58098b55f85baf8698535d3c3c29488deeb89cc8349eed4: Status 404 returned error can't find the container with id cc33593250837cf3a58098b55f85baf8698535d3c3c29488deeb89cc8349eed4 Jun 06 22:06:40.837000 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:40.836969 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:41.166039 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:41.166013 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:06:41.167576 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:41.167552 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fzq8w" event={"ID":"56f37b5d-b920-4c7c-af0e-d35708ff9d95","Type":"ContainerStarted","Data":"0ac7a6afbbb32540eefaa2b06f64c932313eee6614e584dfe8cfe932a99565fe"} Jun 06 22:06:41.167665 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:41.167625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fzq8w" event={"ID":"56f37b5d-b920-4c7c-af0e-d35708ff9d95","Type":"ContainerStarted","Data":"68885e0f0edff4ebc5ac8dd6c6470f24dd9412bf6d690f21cdad13f3fb673174"} Jun 06 22:06:41.167665 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:41.167645 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fzq8w" event={"ID":"56f37b5d-b920-4c7c-af0e-d35708ff9d95","Type":"ContainerStarted","Data":"cc33593250837cf3a58098b55f85baf8698535d3c3c29488deeb89cc8349eed4"} Jun 06 22:06:43.175291 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:43.175255 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fzq8w" event={"ID":"56f37b5d-b920-4c7c-af0e-d35708ff9d95","Type":"ContainerStarted","Data":"a0b703b160e2e50644a115a167946fc56d97e67586526b3ed9d4e357ac7ca4db"} Jun 06 22:06:43.193268 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:43.193220 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fzq8w" podStartSLOduration=1.7333159519999999 podStartE2EDuration="4.193205717s" totalImagesPullingTime="2.459889765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:06:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:43.192301218 +0000 UTC m=+230.191280703" watchObservedRunningTime="2026-06-06 22:06:43.193205717 +0000 UTC m=+230.192185201" Jun 06 22:06:44.387194 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:44.387160 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:44.387600 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:44.387210 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:06:44.387600 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:44.387501 2580 scope.go:117] "RemoveContainer" containerID="71fbf2a2ae221061baf87088d62cf9124eeb6d40a913cbe71544dff62302375d" Jun 06 22:06:44.387666 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:44.387650 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-4ngmg_openshift-console-operator(d78b6164-d3a6-4217-a627-f2e0fb3ade06)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" podUID="d78b6164-d3a6-4217-a627-f2e0fb3ade06" Jun 06 22:06:45.392744 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:45.392708 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:45.493503 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:45.493454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:45.831014 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:45.830922 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:45.833269 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:45.833236 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff32d99a-8480-4abf-91d3-8eb414d4bf85-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-hhv9q\" (UID: \"ff32d99a-8480-4abf-91d3-8eb414d4bf85\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:45.932076 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:45.932039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:45.932620 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:45.932603 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d-service-ca-bundle\") pod \"router-default-78476cbbd5-ldsl9\" (UID: \"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d\") " pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:46.066097 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:46.066058 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" Jun 06 22:06:46.186903 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:46.186838 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q"] Jun 06 22:06:46.189300 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:46.189273 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff32d99a_8480_4abf_91d3_8eb414d4bf85.slice/crio-219934637e14b18c54630cb6316df0e8f949a48d2fe9a73892377681b3112eee WatchSource:0}: Error finding container 219934637e14b18c54630cb6316df0e8f949a48d2fe9a73892377681b3112eee: Status 404 returned error can't find the container with id 219934637e14b18c54630cb6316df0e8f949a48d2fe9a73892377681b3112eee Jun 06 22:06:46.191125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:46.191106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:46.314682 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:46.314652 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78476cbbd5-ldsl9"] Jun 06 22:06:46.317596 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:46.317567 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a9ecc0_7e21_4e62_bf53_e1b3e4320d5d.slice/crio-e5964f1e5ae97ee44a861c96aec7b00353d6b290331684a289462a9381e63239 WatchSource:0}: Error finding container e5964f1e5ae97ee44a861c96aec7b00353d6b290331684a289462a9381e63239: Status 404 returned error can't find the container with id e5964f1e5ae97ee44a861c96aec7b00353d6b290331684a289462a9381e63239 Jun 06 22:06:47.190092 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:47.190049 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78476cbbd5-ldsl9" event={"ID":"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d","Type":"ContainerStarted","Data":"3f825a577ff8fe6f043e9090ff95b68ce702e404b55842b39355334ccdedf9a0"} Jun 06 22:06:47.190598 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:47.190111 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78476cbbd5-ldsl9" event={"ID":"17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d","Type":"ContainerStarted","Data":"e5964f1e5ae97ee44a861c96aec7b00353d6b290331684a289462a9381e63239"} Jun 06 22:06:47.191412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:47.191306 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:47.191412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:47.191371 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" event={"ID":"ff32d99a-8480-4abf-91d3-8eb414d4bf85","Type":"ContainerStarted","Data":"219934637e14b18c54630cb6316df0e8f949a48d2fe9a73892377681b3112eee"} Jun 06 22:06:47.196252 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:47.196227 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:47.210402 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:47.210350 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-78476cbbd5-ldsl9" podStartSLOduration=33.210329076 podStartE2EDuration="33.210329076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:47.208754057 +0000 UTC m=+234.207733543" watchObservedRunningTime="2026-06-06 22:06:47.210329076 +0000 UTC m=+234.209308597" Jun 06 22:06:48.195717 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.195681 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" event={"ID":"ff32d99a-8480-4abf-91d3-8eb414d4bf85","Type":"ContainerStarted","Data":"f9e7e6004fcd71477a94d78d0cb01b0e20097fb86314a940cddef6a3673ebd9c"} Jun 06 22:06:48.196191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.195934 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:48.197027 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.197004 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-78476cbbd5-ldsl9" Jun 06 22:06:48.209932 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.209893 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-hhv9q" podStartSLOduration=33.963872799 podStartE2EDuration="35.209880222s" totalImagesPullingTime="1.246007423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:48.209184051 +0000 UTC m=+235.208163535" watchObservedRunningTime="2026-06-06 22:06:48.209880222 +0000 UTC m=+235.208859706" Jun 06 22:06:48.322333 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.322305 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:48.422995 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.422961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:48.492517 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.492445 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6"] Jun 06 22:06:48.495668 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.495652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" Jun 06 22:06:48.498137 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.498119 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 06 22:06:48.498218 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.498202 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9r6sf\"" Jun 06 22:06:48.500516 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.500489 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:48.503879 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.503857 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6"] Jun 06 22:06:48.549291 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.549259 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/819aa35f-218e-4f27-b32f-6d3d4adc9e26-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-rvvb6\" (UID: \"819aa35f-218e-4f27-b32f-6d3d4adc9e26\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" Jun 06 22:06:48.626403 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.626368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:48.650619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.650589 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/819aa35f-218e-4f27-b32f-6d3d4adc9e26-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-rvvb6\" (UID: \"819aa35f-218e-4f27-b32f-6d3d4adc9e26\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" Jun 06 22:06:48.652940 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.652923 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/819aa35f-218e-4f27-b32f-6d3d4adc9e26-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-rvvb6\" (UID: \"819aa35f-218e-4f27-b32f-6d3d4adc9e26\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" Jun 06 22:06:48.806021 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.805924 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" Jun 06 22:06:48.926769 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:48.926664 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6"] Jun 06 22:06:48.929141 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:48.929115 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819aa35f_218e_4f27_b32f_6d3d4adc9e26.slice/crio-0c774b48c615c95da327758e0fe3a822622f50ac83de2f9de5f6dc3654359273 WatchSource:0}: Error finding container 0c774b48c615c95da327758e0fe3a822622f50ac83de2f9de5f6dc3654359273: Status 404 returned error can't find the container with id 0c774b48c615c95da327758e0fe3a822622f50ac83de2f9de5f6dc3654359273 Jun 06 22:06:49.200901 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:49.200860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" event={"ID":"819aa35f-218e-4f27-b32f-6d3d4adc9e26","Type":"ContainerStarted","Data":"0c774b48c615c95da327758e0fe3a822622f50ac83de2f9de5f6dc3654359273"} Jun 06 22:06:50.204956 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:50.204917 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" event={"ID":"819aa35f-218e-4f27-b32f-6d3d4adc9e26","Type":"ContainerStarted","Data":"ce73b018e06b7a3f4766189356bcdf65e442c56aa3dbd54a10017921a27bed56"} Jun 06 22:06:50.220357 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:50.220289 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" podStartSLOduration=1.320017248 podStartE2EDuration="2.220274848s" totalImagesPullingTime="900.2576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:50.219547259 +0000 UTC m=+237.218526743" watchObservedRunningTime="2026-06-06 22:06:50.220274848 +0000 UTC m=+237.219254331" Jun 06 22:06:51.209321 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.209277 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" Jun 06 22:06:51.213735 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.213711 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-rvvb6" Jun 06 22:06:51.500495 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.500416 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:51.542226 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.542203 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd"] Jun 06 22:06:51.545346 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.545319 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.549026 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.548980 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 06 22:06:51.549026 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.548988 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 06 22:06:51.549184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.549057 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 06 22:06:51.549184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.549012 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 06 22:06:51.549184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.549165 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 06 22:06:51.549347 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.549312 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-w4cfd\"" Jun 06 22:06:51.553671 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.553651 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd"] Jun 06 22:06:51.558750 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.558725 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:51.570947 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.570920 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b815c5c4-8fee-4374-a16f-8bf93b94d346-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.571057 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.570962 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn6z8\" (UniqueName: \"kubernetes.io/projected/b815c5c4-8fee-4374-a16f-8bf93b94d346-kube-api-access-vn6z8\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.571057 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.571023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b815c5c4-8fee-4374-a16f-8bf93b94d346-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.571057 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.571044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b815c5c4-8fee-4374-a16f-8bf93b94d346-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.671528 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.671496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b815c5c4-8fee-4374-a16f-8bf93b94d346-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.671672 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.671539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b815c5c4-8fee-4374-a16f-8bf93b94d346-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.671672 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.671582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn6z8\" (UniqueName: \"kubernetes.io/projected/b815c5c4-8fee-4374-a16f-8bf93b94d346-kube-api-access-vn6z8\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.671672 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.671653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b815c5c4-8fee-4374-a16f-8bf93b94d346-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.671865 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:51.671696 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 06 22:06:51.671865 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:51.671762 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b815c5c4-8fee-4374-a16f-8bf93b94d346-prometheus-operator-tls podName:b815c5c4-8fee-4374-a16f-8bf93b94d346 nodeName:}" failed. No retries permitted until 2026-06-06 22:06:52.171744445 +0000 UTC m=+239.170723907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b815c5c4-8fee-4374-a16f-8bf93b94d346-prometheus-operator-tls") pod "prometheus-operator-7f7d445d84-5mdqd" (UID: "b815c5c4-8fee-4374-a16f-8bf93b94d346") : secret "prometheus-operator-tls" not found Jun 06 22:06:51.672315 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.672297 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b815c5c4-8fee-4374-a16f-8bf93b94d346-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.673912 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.673894 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b815c5c4-8fee-4374-a16f-8bf93b94d346-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:51.681900 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:51.681882 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn6z8\" (UniqueName: \"kubernetes.io/projected/b815c5c4-8fee-4374-a16f-8bf93b94d346-kube-api-access-vn6z8\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:52.049393 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:52.049360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:52.175764 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:52.175720 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b815c5c4-8fee-4374-a16f-8bf93b94d346-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:52.178074 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:52.178055 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b815c5c4-8fee-4374-a16f-8bf93b94d346-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-5mdqd\" (UID: \"b815c5c4-8fee-4374-a16f-8bf93b94d346\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:52.455411 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:52.455378 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" Jun 06 22:06:52.576664 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:52.576622 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd"] Jun 06 22:06:52.579530 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:52.579505 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb815c5c4_8fee_4374_a16f_8bf93b94d346.slice/crio-82cc633a27dcc177e45153fa9843b61cd487bee8c8c695f6391e006394900eff WatchSource:0}: Error finding container 82cc633a27dcc177e45153fa9843b61cd487bee8c8c695f6391e006394900eff: Status 404 returned error can't find the container with id 82cc633a27dcc177e45153fa9843b61cd487bee8c8c695f6391e006394900eff Jun 06 22:06:53.218619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:53.218588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" event={"ID":"b815c5c4-8fee-4374-a16f-8bf93b94d346","Type":"ContainerStarted","Data":"82cc633a27dcc177e45153fa9843b61cd487bee8c8c695f6391e006394900eff"} Jun 06 22:06:53.946690 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:53.946651 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:54.223897 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:54.223858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" event={"ID":"b815c5c4-8fee-4374-a16f-8bf93b94d346","Type":"ContainerStarted","Data":"d5caa9bce42c98ba941ed627166e25e9623e8488506fa5f528d5df91be72b9d1"} Jun 06 22:06:54.224044 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:54.223941 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" event={"ID":"b815c5c4-8fee-4374-a16f-8bf93b94d346","Type":"ContainerStarted","Data":"c11596d6ff8476799afcc4c0637cf0bdfc7c4ce6beacad3ed1fa566f4320c51a"} Jun 06 22:06:54.242266 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:54.242220 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-5mdqd" podStartSLOduration=2.098612103 podStartE2EDuration="3.242206716s" totalImagesPullingTime="1.143594613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:54.241096803 +0000 UTC m=+241.240076287" watchObservedRunningTime="2026-06-06 22:06:54.242206716 +0000 UTC m=+241.241186199" Jun 06 22:06:55.649778 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.649746 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:55.856890 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.856852 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:55.881202 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.881170 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fbrjj"] Jun 06 22:06:55.885093 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.885060 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:55.887617 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.887594 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 06 22:06:55.887757 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.887736 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 06 22:06:55.887860 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.887606 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l4vd7\"" Jun 06 22:06:55.888108 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.888088 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 06 22:06:55.895039 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.895009 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6"] Jun 06 22:06:55.898217 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.898199 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:55.900503 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.900438 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 06 22:06:55.901090 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.901069 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 06 22:06:55.901278 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.901261 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kjcz2\"" Jun 06 22:06:55.901413 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.901397 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 06 22:06:55.912232 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.912209 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6"] Jun 06 22:06:55.966544 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:55.966513 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:56.006710 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.006677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.006907 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.006729 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2294bbdf-04ad-4961-8d58-01f80a5464f2-sys\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.006907 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.006795 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-textfile\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.006907 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.006873 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2294bbdf-04ad-4961-8d58-01f80a5464f2-root\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.007045 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.006905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.007045 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.006942 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.007045 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.006968 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.007045 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.007005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-tls\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.007045 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.007034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-wtmp\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.007214 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.007056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.007214 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.007110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.007214 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.007156 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb6ks\" (UniqueName: \"kubernetes.io/projected/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-api-access-bb6ks\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.007214 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.007204 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2294bbdf-04ad-4961-8d58-01f80a5464f2-metrics-client-ca\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.007358 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.007230 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcqzw\" (UniqueName: \"kubernetes.io/projected/2294bbdf-04ad-4961-8d58-01f80a5464f2-kube-api-access-hcqzw\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.007358 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.007254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.031296 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.031261 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:56.108583 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108540 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-textfile\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.108742 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108595 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2294bbdf-04ad-4961-8d58-01f80a5464f2-root\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.108742 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.108742 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.108742 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108664 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2294bbdf-04ad-4961-8d58-01f80a5464f2-root\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.108742 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108686 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.108742 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-tls\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-wtmp\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108833 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb6ks\" (UniqueName: \"kubernetes.io/projected/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-api-access-bb6ks\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2294bbdf-04ad-4961-8d58-01f80a5464f2-metrics-client-ca\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcqzw\" (UniqueName: \"kubernetes.io/projected/2294bbdf-04ad-4961-8d58-01f80a5464f2-kube-api-access-hcqzw\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108982 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108992 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-textfile\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.108997 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-wtmp\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.109017 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.109086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.109063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2294bbdf-04ad-4961-8d58-01f80a5464f2-sys\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.109597 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.109124 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2294bbdf-04ad-4961-8d58-01f80a5464f2-sys\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.109647 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.109597 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.110012 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.109986 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.110102 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.109998 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.110350 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.110305 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2294bbdf-04ad-4961-8d58-01f80a5464f2-metrics-client-ca\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.110538 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.110504 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.112500 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.112160 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.112500 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.112255 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.113003 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.112981 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.118364 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.118340 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb6ks\" (UniqueName: \"kubernetes.io/projected/40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764-kube-api-access-bb6ks\") pod \"kube-state-metrics-57bbf8bfb5-dzfv6\" (UID: \"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.118921 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.118902 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcqzw\" (UniqueName: \"kubernetes.io/projected/2294bbdf-04ad-4961-8d58-01f80a5464f2-kube-api-access-hcqzw\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.119166 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.119046 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2294bbdf-04ad-4961-8d58-01f80a5464f2-node-exporter-tls\") pod \"node-exporter-fbrjj\" (UID: \"2294bbdf-04ad-4961-8d58-01f80a5464f2\") " pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.131757 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.131635 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:56.197534 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.197450 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fbrjj" Jun 06 22:06:56.209679 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.209655 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" Jun 06 22:06:56.210213 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:56.210187 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2294bbdf_04ad_4961_8d58_01f80a5464f2.slice/crio-617cc69c005be4d3b1b2f966b5272f9bc372a13d8061aeaf64c090c2aa8f699b WatchSource:0}: Error finding container 617cc69c005be4d3b1b2f966b5272f9bc372a13d8061aeaf64c090c2aa8f699b: Status 404 returned error can't find the container with id 617cc69c005be4d3b1b2f966b5272f9bc372a13d8061aeaf64c090c2aa8f699b Jun 06 22:06:56.231629 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.231601 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:56.232689 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.232663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbrjj" event={"ID":"2294bbdf-04ad-4961-8d58-01f80a5464f2","Type":"ContainerStarted","Data":"617cc69c005be4d3b1b2f966b5272f9bc372a13d8061aeaf64c090c2aa8f699b"} Jun 06 22:06:56.331702 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.331673 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:56.360190 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.360161 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6"] Jun 06 22:06:56.365302 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:06:56.365264 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40bb3b22_4cfb_4bf3_863e_ac1ffdfc3764.slice/crio-25c19e5af8bff9f0338c480686f24e88bce67cf0baf757407fcfbfb9f7f6c80d WatchSource:0}: Error finding container 25c19e5af8bff9f0338c480686f24e88bce67cf0baf757407fcfbfb9f7f6c80d: Status 404 returned error can't find the container with id 25c19e5af8bff9f0338c480686f24e88bce67cf0baf757407fcfbfb9f7f6c80d Jun 06 22:06:56.618726 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.618640 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:56.719076 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.719044 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:56.870577 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.870471 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:56.944047 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:56.943893 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:57.238149 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:57.238123 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" event={"ID":"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764","Type":"ContainerStarted","Data":"25c19e5af8bff9f0338c480686f24e88bce67cf0baf757407fcfbfb9f7f6c80d"} Jun 06 22:06:57.564099 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:57.564006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:58.243165 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:58.243131 2580 generic.go:356] "Generic (PLEG): container finished" podID="2294bbdf-04ad-4961-8d58-01f80a5464f2" containerID="029b858d5819414c0623a92c95dafef32e461f5e5a3c6010a1b051e37720eeb0" exitCode=0 Jun 06 22:06:58.243579 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:58.243218 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbrjj" event={"ID":"2294bbdf-04ad-4961-8d58-01f80a5464f2","Type":"ContainerDied","Data":"029b858d5819414c0623a92c95dafef32e461f5e5a3c6010a1b051e37720eeb0"} Jun 06 22:06:58.245301 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:58.245279 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" event={"ID":"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764","Type":"ContainerStarted","Data":"fff7fa5249ffd708d190b6a6d9f59d9068e77f882e7c4f660ea7b11aafe83d17"} Jun 06 22:06:58.245412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:58.245329 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" event={"ID":"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764","Type":"ContainerStarted","Data":"fd368cab58e52d051f13e59f087e3caada6208be851c796a2f59c607640e9948"} Jun 06 22:06:58.245412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:58.245364 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" event={"ID":"40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764","Type":"ContainerStarted","Data":"379a345b63ccb1de3597fc8b9762e0b210ed6a5e71b4dc92cb96bd1a1662ddea"} Jun 06 22:06:58.276773 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:58.276728 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-dzfv6" podStartSLOduration=2.02209098 podStartE2EDuration="3.276711642s" totalImagesPullingTime="1.254620662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:58.276058626 +0000 UTC m=+245.275038167" watchObservedRunningTime="2026-06-06 22:06:58.276711642 +0000 UTC m=+245.275691126" Jun 06 22:06:58.483397 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:58.483370 2580 scope.go:117] "RemoveContainer" containerID="71fbf2a2ae221061baf87088d62cf9124eeb6d40a913cbe71544dff62302375d" Jun 06 22:06:58.483557 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:06:58.483533 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-4ngmg_openshift-console-operator(d78b6164-d3a6-4217-a627-f2e0fb3ade06)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" podUID="d78b6164-d3a6-4217-a627-f2e0fb3ade06" Jun 06 22:06:58.807904 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:58.807858 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:58.859418 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:58.859387 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:59.061408 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:59.061321 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:59.200340 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:59.200304 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:06:59.251122 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:59.251079 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbrjj" event={"ID":"2294bbdf-04ad-4961-8d58-01f80a5464f2","Type":"ContainerStarted","Data":"3ad8fdcc43aec25d6e72f194dd326a979c09dfc903c8b6d894c4b26dd1b0ba12"} Jun 06 22:06:59.251517 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:59.251142 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fbrjj" event={"ID":"2294bbdf-04ad-4961-8d58-01f80a5464f2","Type":"ContainerStarted","Data":"d5860490cbecd683f8fe2135934de3917523ee8cc951838dbe93007e5f39ef6d"} Jun 06 22:06:59.270401 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:06:59.270353 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fbrjj" podStartSLOduration=3.207910835 podStartE2EDuration="4.270336572s" totalImagesPullingTime="987.815681ms" totalInitContainerRuntime="74.610056ms" isStatefulPod=true podCreationTimestamp="2026-06-06 22:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:06:59.268666662 +0000 UTC m=+246.267646167" watchObservedRunningTime="2026-06-06 22:06:59.270336572 +0000 UTC m=+246.269316082" Jun 06 22:07:00.196011 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.195973 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5fc4968dbb-jzvwh"] Jun 06 22:07:00.199220 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.199199 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.202141 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.202121 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 06 22:07:00.202278 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.202261 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-w6mdn\"" Jun 06 22:07:00.202328 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.202316 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 06 22:07:00.203189 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.203167 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 06 22:07:00.203325 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.203309 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 06 22:07:00.203410 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.203381 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-aerorb1tfeifu\"" Jun 06 22:07:00.209013 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.208992 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fc4968dbb-jzvwh"] Jun 06 22:07:00.239954 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.239923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6096f400-3aba-4410-bf60-7ae2b9bdb31e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.240103 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.239967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4jwb\" (UniqueName: \"kubernetes.io/projected/6096f400-3aba-4410-bf60-7ae2b9bdb31e-kube-api-access-b4jwb\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.240103 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.239999 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6096f400-3aba-4410-bf60-7ae2b9bdb31e-metrics-server-audit-profiles\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.240103 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.240058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6096f400-3aba-4410-bf60-7ae2b9bdb31e-secret-metrics-server-tls\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.240285 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.240147 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6096f400-3aba-4410-bf60-7ae2b9bdb31e-secret-metrics-server-client-certs\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.240285 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.240179 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6096f400-3aba-4410-bf60-7ae2b9bdb31e-audit-log\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.240285 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.240215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6096f400-3aba-4410-bf60-7ae2b9bdb31e-client-ca-bundle\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.341165 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.341127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6096f400-3aba-4410-bf60-7ae2b9bdb31e-client-ca-bundle\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.341601 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.341180 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6096f400-3aba-4410-bf60-7ae2b9bdb31e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.341601 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.341209 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4jwb\" (UniqueName: \"kubernetes.io/projected/6096f400-3aba-4410-bf60-7ae2b9bdb31e-kube-api-access-b4jwb\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.341601 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.341249 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6096f400-3aba-4410-bf60-7ae2b9bdb31e-metrics-server-audit-profiles\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.341601 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.341269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6096f400-3aba-4410-bf60-7ae2b9bdb31e-secret-metrics-server-tls\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.341601 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.341325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6096f400-3aba-4410-bf60-7ae2b9bdb31e-secret-metrics-server-client-certs\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.341910 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.341856 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6096f400-3aba-4410-bf60-7ae2b9bdb31e-audit-log\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.342108 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.342080 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6096f400-3aba-4410-bf60-7ae2b9bdb31e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.342236 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.342216 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6096f400-3aba-4410-bf60-7ae2b9bdb31e-audit-log\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.342379 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.342357 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6096f400-3aba-4410-bf60-7ae2b9bdb31e-metrics-server-audit-profiles\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.343764 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.343740 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6096f400-3aba-4410-bf60-7ae2b9bdb31e-secret-metrics-server-tls\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.343888 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.343793 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6096f400-3aba-4410-bf60-7ae2b9bdb31e-client-ca-bundle\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.343985 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.343966 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6096f400-3aba-4410-bf60-7ae2b9bdb31e-secret-metrics-server-client-certs\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.349186 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.349164 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4jwb\" (UniqueName: \"kubernetes.io/projected/6096f400-3aba-4410-bf60-7ae2b9bdb31e-kube-api-access-b4jwb\") pod \"metrics-server-5fc4968dbb-jzvwh\" (UID: \"6096f400-3aba-4410-bf60-7ae2b9bdb31e\") " pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.394011 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.393971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:00.465408 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.465328 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:00.507965 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.507935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:00.637291 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.637216 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5fc4968dbb-jzvwh"] Jun 06 22:07:00.642928 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:07:00.642882 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6096f400_3aba_4410_bf60_7ae2b9bdb31e.slice/crio-0673ca99caccddef98e19ff9989853977abc4b09cc32f92ba3df7f247caefeeb WatchSource:0}: Error finding container 0673ca99caccddef98e19ff9989853977abc4b09cc32f92ba3df7f247caefeeb: Status 404 returned error can't find the container with id 0673ca99caccddef98e19ff9989853977abc4b09cc32f92ba3df7f247caefeeb Jun 06 22:07:00.663761 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.663733 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc"] Jun 06 22:07:00.669344 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.669320 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:00.672069 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.672047 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 06 22:07:00.672175 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.672078 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-94xzm\"" Jun 06 22:07:00.676543 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.676481 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc"] Jun 06 22:07:00.678017 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.677889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:00.745973 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.745881 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06e6a9e9-c439-423f-98da-80a7e0f105ce-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-8pfnc\" (UID: \"06e6a9e9-c439-423f-98da-80a7e0f105ce\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:00.745973 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.745939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfrb5\" (UniqueName: \"kubernetes.io/projected/06e6a9e9-c439-423f-98da-80a7e0f105ce-kube-api-access-bfrb5\") pod \"monitoring-plugin-76b68cc874-8pfnc\" (UID: \"06e6a9e9-c439-423f-98da-80a7e0f105ce\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:00.847342 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.847307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06e6a9e9-c439-423f-98da-80a7e0f105ce-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-8pfnc\" (UID: \"06e6a9e9-c439-423f-98da-80a7e0f105ce\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:00.847499 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.847364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfrb5\" (UniqueName: \"kubernetes.io/projected/06e6a9e9-c439-423f-98da-80a7e0f105ce-kube-api-access-bfrb5\") pod \"monitoring-plugin-76b68cc874-8pfnc\" (UID: \"06e6a9e9-c439-423f-98da-80a7e0f105ce\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:00.847499 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:07:00.847447 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 06 22:07:00.847627 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:07:00.847513 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06e6a9e9-c439-423f-98da-80a7e0f105ce-monitoring-plugin-cert podName:06e6a9e9-c439-423f-98da-80a7e0f105ce nodeName:}" failed. No retries permitted until 2026-06-06 22:07:01.347497365 +0000 UTC m=+248.346476830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/06e6a9e9-c439-423f-98da-80a7e0f105ce-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-8pfnc" (UID: "06e6a9e9-c439-423f-98da-80a7e0f105ce") : secret "monitoring-plugin-cert" not found Jun 06 22:07:00.856170 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:00.856141 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfrb5\" (UniqueName: \"kubernetes.io/projected/06e6a9e9-c439-423f-98da-80a7e0f105ce-kube-api-access-bfrb5\") pod \"monitoring-plugin-76b68cc874-8pfnc\" (UID: \"06e6a9e9-c439-423f-98da-80a7e0f105ce\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:01.072234 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.072151 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:01.118221 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.118187 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc"] Jun 06 22:07:01.121544 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.121522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.124087 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.124067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 06 22:07:01.124395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.124377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 06 22:07:01.124471 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.124449 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-npjpf\"" Jun 06 22:07:01.124521 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.124468 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 06 22:07:01.124603 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.124583 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 06 22:07:01.124770 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.124756 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 06 22:07:01.129104 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.129081 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 06 22:07:01.133399 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.133375 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc"] Jun 06 22:07:01.251241 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.251205 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79a9d2e2-54a3-4dca-b503-0a8110954c69-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.251442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.251358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79a9d2e2-54a3-4dca-b503-0a8110954c69-serving-certs-ca-bundle\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.251442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.251391 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79a9d2e2-54a3-4dca-b503-0a8110954c69-metrics-client-ca\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.251442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.251428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.251613 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.251479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj2g4\" (UniqueName: \"kubernetes.io/projected/79a9d2e2-54a3-4dca-b503-0a8110954c69-kube-api-access-sj2g4\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.251613 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.251500 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-federate-client-tls\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.251613 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.251518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-secret-telemeter-client\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.251613 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.251534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-telemeter-client-tls\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.259842 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.259792 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" event={"ID":"6096f400-3aba-4410-bf60-7ae2b9bdb31e","Type":"ContainerStarted","Data":"0673ca99caccddef98e19ff9989853977abc4b09cc32f92ba3df7f247caefeeb"} Jun 06 22:07:01.352619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.352531 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79a9d2e2-54a3-4dca-b503-0a8110954c69-serving-certs-ca-bundle\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.352619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.352563 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79a9d2e2-54a3-4dca-b503-0a8110954c69-metrics-client-ca\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.352619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.352589 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.353237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.352720 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06e6a9e9-c439-423f-98da-80a7e0f105ce-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-8pfnc\" (UID: \"06e6a9e9-c439-423f-98da-80a7e0f105ce\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:01.353237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.352757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj2g4\" (UniqueName: \"kubernetes.io/projected/79a9d2e2-54a3-4dca-b503-0a8110954c69-kube-api-access-sj2g4\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.353237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.352775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-federate-client-tls\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.353237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.352825 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-secret-telemeter-client\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.353237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.352855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-telemeter-client-tls\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.353237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.352912 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79a9d2e2-54a3-4dca-b503-0a8110954c69-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.353552 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.353419 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79a9d2e2-54a3-4dca-b503-0a8110954c69-metrics-client-ca\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.353552 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.353474 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79a9d2e2-54a3-4dca-b503-0a8110954c69-serving-certs-ca-bundle\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.353796 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.353773 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79a9d2e2-54a3-4dca-b503-0a8110954c69-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.355378 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.355349 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06e6a9e9-c439-423f-98da-80a7e0f105ce-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-8pfnc\" (UID: \"06e6a9e9-c439-423f-98da-80a7e0f105ce\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:01.355518 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.355501 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.355824 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.355779 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-secret-telemeter-client\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.355889 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.355831 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-telemeter-client-tls\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.355889 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.355846 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/79a9d2e2-54a3-4dca-b503-0a8110954c69-federate-client-tls\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.360265 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.360239 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj2g4\" (UniqueName: \"kubernetes.io/projected/79a9d2e2-54a3-4dca-b503-0a8110954c69-kube-api-access-sj2g4\") pod \"telemeter-client-6c97f86fb6-jpmvc\" (UID: \"79a9d2e2-54a3-4dca-b503-0a8110954c69\") " pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.431560 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.431524 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" Jun 06 22:07:01.580813 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.580780 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:01.593814 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.593776 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc"] Jun 06 22:07:01.595645 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:07:01.595619 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a9d2e2_54a3_4dca_b503_0a8110954c69.slice/crio-58b97daa04c76ec435b9faedbc86686c1e000f8847eb13c6cce1271859e76e9f WatchSource:0}: Error finding container 58b97daa04c76ec435b9faedbc86686c1e000f8847eb13c6cce1271859e76e9f: Status 404 returned error can't find the container with id 58b97daa04c76ec435b9faedbc86686c1e000f8847eb13c6cce1271859e76e9f Jun 06 22:07:01.715073 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:01.715037 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc"] Jun 06 22:07:01.719845 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:07:01.719784 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e6a9e9_c439_423f_98da_80a7e0f105ce.slice/crio-60c1b30dc5c54142b598a64251d687bb1b198ca6326b03ca4e699b3c612f74ad WatchSource:0}: Error finding container 60c1b30dc5c54142b598a64251d687bb1b198ca6326b03ca4e699b3c612f74ad: Status 404 returned error can't find the container with id 60c1b30dc5c54142b598a64251d687bb1b198ca6326b03ca4e699b3c612f74ad Jun 06 22:07:02.082391 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.082351 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 06 22:07:02.086204 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.086169 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:02.087835 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.087790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.091272 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.090987 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-co2uctam9c9oc\"" Jun 06 22:07:02.091272 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.091056 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 06 22:07:02.091272 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.091089 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 06 22:07:02.091272 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.091100 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b896k\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.092191 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.092261 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.092432 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.092516 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.092191 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.092885 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.093088 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.093286 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.093775 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 06 22:07:02.094231 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.094018 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 06 22:07:02.094767 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.094702 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 06 22:07:02.095126 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.094975 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 06 22:07:02.103576 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.103388 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 06 22:07:02.161232 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161409 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddg4n\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-kube-api-access-ddg4n\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161409 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161326 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161409 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161571 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161571 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161571 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161696 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161696 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161783 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161697 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161783 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161721 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161783 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161748 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161959 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161793 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161959 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161832 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161959 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161902 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.161959 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.162140 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.161971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.162140 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.162010 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.162140 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.162036 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.162140 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.162108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.264978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.264926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266038 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266156 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266041 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266156 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266156 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266634 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266695 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.266780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.267184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.267184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.267184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266870 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.267184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddg4n\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-kube-api-access-ddg4n\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.267184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.267184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.266987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.267184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.267013 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.267184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.267053 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.267184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.267090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.269526 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.269495 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.269772 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.269751 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.270541 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.270115 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.270541 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.270161 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.270541 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.270185 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.270541 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.270423 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.270832 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.270797 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.271994 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.271076 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.271994 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.271505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.271994 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.271875 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.274878 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.274354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.275627 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.275119 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.275627 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.275612 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.277526 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.277500 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.278010 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.277826 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.278173 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.278146 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.278376 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.278354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.280155 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.280128 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" event={"ID":"06e6a9e9-c439-423f-98da-80a7e0f105ce","Type":"ContainerStarted","Data":"60c1b30dc5c54142b598a64251d687bb1b198ca6326b03ca4e699b3c612f74ad"} Jun 06 22:07:02.281713 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.281689 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" event={"ID":"79a9d2e2-54a3-4dca-b503-0a8110954c69","Type":"ContainerStarted","Data":"58b97daa04c76ec435b9faedbc86686c1e000f8847eb13c6cce1271859e76e9f"} Jun 06 22:07:02.285953 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.285327 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.286641 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.286617 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddg4n\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-kube-api-access-ddg4n\") pod \"prometheus-k8s-0\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.400952 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.400923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:02.593275 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:02.593096 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 06 22:07:03.287406 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:03.287369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerStarted","Data":"ff8ed26e6449f370cdb0f20d8931aff62f4f9defdf6e5335f7d5107888c51ab9"} Jun 06 22:07:03.288968 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:03.288937 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" event={"ID":"6096f400-3aba-4410-bf60-7ae2b9bdb31e","Type":"ContainerStarted","Data":"723760fcf32f894e03024dfd89d251343cc5f4996daf9c865233abfee760d4f6"} Jun 06 22:07:03.307625 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:03.307564 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" podStartSLOduration=1.647496372 podStartE2EDuration="3.30754172s" totalImagesPullingTime="1.660045348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:07:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:03.306165394 +0000 UTC m=+250.305144878" watchObservedRunningTime="2026-06-06 22:07:03.30754172 +0000 UTC m=+250.306521206" Jun 06 22:07:04.294203 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.294168 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" event={"ID":"06e6a9e9-c439-423f-98da-80a7e0f105ce","Type":"ContainerStarted","Data":"ab35c5a991966e1fcf495264e63a2aeeac3819114b9a71f2cdd6dd3af8ac0342"} Jun 06 22:07:04.294571 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.294234 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:04.296475 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.296435 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" event={"ID":"79a9d2e2-54a3-4dca-b503-0a8110954c69","Type":"ContainerStarted","Data":"ceaa40607960e75f5d1ab0add2ebbaa2f9924690ae538bdc9fa50b219da42772"} Jun 06 22:07:04.296574 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.296498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" event={"ID":"79a9d2e2-54a3-4dca-b503-0a8110954c69","Type":"ContainerStarted","Data":"d544c3440eaefae9e380eefb81020a3fcc2044724a6e4f42b0ebab27dae5e576"} Jun 06 22:07:04.296574 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.296545 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" event={"ID":"79a9d2e2-54a3-4dca-b503-0a8110954c69","Type":"ContainerStarted","Data":"fd395eb949729596a3e7267a81885bf26ab7e1e7dff24a6c018defb56e0ce9d9"} Jun 06 22:07:04.298080 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.298054 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerID="a98106c357295708d33a1b309616a3e47877ebdad334e7d2eceadd180fdf131c" exitCode=0 Jun 06 22:07:04.298182 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.298078 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerDied","Data":"a98106c357295708d33a1b309616a3e47877ebdad334e7d2eceadd180fdf131c"} Jun 06 22:07:04.300258 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.300237 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" Jun 06 22:07:04.309102 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.309062 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-8pfnc" podStartSLOduration=1.983708457 podStartE2EDuration="4.309048103s" totalImagesPullingTime="2.325339646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:07:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:04.307603574 +0000 UTC m=+251.306583059" watchObservedRunningTime="2026-06-06 22:07:04.309048103 +0000 UTC m=+251.308027586" Jun 06 22:07:04.345371 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:04.345319 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6c97f86fb6-jpmvc" podStartSLOduration=0.890376211 podStartE2EDuration="3.345296271s" totalImagesPullingTime="2.45492006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:04.344252224 +0000 UTC m=+251.343231718" watchObservedRunningTime="2026-06-06 22:07:04.345296271 +0000 UTC m=+251.344275779" Jun 06 22:07:05.298667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:05.298627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:07:05.301878 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:05.301797 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4eec8c-1779-4a2f-b0e0-a028da53560b-metrics-certs\") pod \"network-metrics-daemon-j7ksf\" (UID: \"5f4eec8c-1779-4a2f-b0e0-a028da53560b\") " pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:07:05.387455 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:05.387367 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d4tj5\"" Jun 06 22:07:05.394894 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:05.394856 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7ksf" Jun 06 22:07:05.569975 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:05.569895 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j7ksf"] Jun 06 22:07:05.572688 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:07:05.572656 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4eec8c_1779_4a2f_b0e0_a028da53560b.slice/crio-d3a07df137906e5e3cdcc3e7cbaf4c5c50a33c6b56be9119a55d06034eb1c8e3 WatchSource:0}: Error finding container d3a07df137906e5e3cdcc3e7cbaf4c5c50a33c6b56be9119a55d06034eb1c8e3: Status 404 returned error can't find the container with id d3a07df137906e5e3cdcc3e7cbaf4c5c50a33c6b56be9119a55d06034eb1c8e3 Jun 06 22:07:06.309692 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:06.309657 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j7ksf" event={"ID":"5f4eec8c-1779-4a2f-b0e0-a028da53560b","Type":"ContainerStarted","Data":"d3a07df137906e5e3cdcc3e7cbaf4c5c50a33c6b56be9119a55d06034eb1c8e3"} Jun 06 22:07:07.251229 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:07.251125 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:07.304666 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:07.304596 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:08.323545 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:08.323500 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j7ksf" event={"ID":"5f4eec8c-1779-4a2f-b0e0-a028da53560b","Type":"ContainerStarted","Data":"21eca5d1e14da0a4c4639ac07c9007e1d6164fece34dacd2c5d65cf745c4a07d"} Jun 06 22:07:08.324002 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:08.323575 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j7ksf" event={"ID":"5f4eec8c-1779-4a2f-b0e0-a028da53560b","Type":"ContainerStarted","Data":"bd6c601d48e71225fa9855e383635b153eca71fae6b975962a6336b1822b686a"} Jun 06 22:07:08.326027 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:08.325995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerStarted","Data":"4c7bf733d77ca7a880277d8fa0723f5f278c67afc4ad098eff3d3df7aef5825b"} Jun 06 22:07:08.326125 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:08.326067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerStarted","Data":"4084830a8108c96f99f015e796118733d7e11b9069f1233d9b090a3816e1ad5e"} Jun 06 22:07:08.341056 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:08.340999 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j7ksf" podStartSLOduration=253.281242523 podStartE2EDuration="4m15.340978632s" totalImagesPullingTime="2.059736109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:08.338746421 +0000 UTC m=+255.337725907" watchObservedRunningTime="2026-06-06 22:07:08.340978632 +0000 UTC m=+255.339958118" Jun 06 22:07:08.422144 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:08.422089 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:08.474694 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:08.474647 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:10.338502 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:10.338465 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerStarted","Data":"e751aa900e5627d7477e880a0b52941028512555a95e04e4469e03ccdf049626"} Jun 06 22:07:10.338892 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:10.338543 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerStarted","Data":"64aeb10741188deb79e15f8234545dfa9dfdbdaa95bf7b5a8d0b7de31aadc2ca"} Jun 06 22:07:10.338892 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:10.338585 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerStarted","Data":"de597d9b65b481ba528e597b52c29fe4da68c5bff6f3bdaea51b1ce7bcf777dd"} Jun 06 22:07:10.338892 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:10.338624 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerStarted","Data":"48b2d0644a9e5ed59ee5727fde91caf20a2ce423dc69de98d9fef8d597a1e5a4"} Jun 06 22:07:10.369075 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:10.369025 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.8306443749999999 podStartE2EDuration="8.369009048s" totalImagesPullingTime="6.517641198s" totalInitContainerRuntime="20.723475ms" isStatefulPod=false podCreationTimestamp="2026-06-06 22:07:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:10.365736278 +0000 UTC m=+257.364715761" watchObservedRunningTime="2026-06-06 22:07:10.369009048 +0000 UTC m=+257.367988531" Jun 06 22:07:12.401142 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:12.401105 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:07:12.483373 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:12.483344 2580 scope.go:117] "RemoveContainer" containerID="71fbf2a2ae221061baf87088d62cf9124eeb6d40a913cbe71544dff62302375d" Jun 06 22:07:13.278454 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.278422 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:13.352066 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.352039 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:07:13.352220 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.352112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" event={"ID":"d78b6164-d3a6-4217-a627-f2e0fb3ade06","Type":"ContainerStarted","Data":"29e7d27b8efbdf4937e74869cc8d705ac61f8a6bf2491ac10b6d1d5fbe543016"} Jun 06 22:07:13.352415 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.352395 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:07:13.357141 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.357117 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" Jun 06 22:07:13.368619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.368576 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-668fdbf85c-4ngmg" podStartSLOduration=56.88096547 podStartE2EDuration="59.368561538s" totalImagesPullingTime="2.487596068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:13.367644739 +0000 UTC m=+260.366624239" watchObservedRunningTime="2026-06-06 22:07:13.368561538 +0000 UTC m=+260.367541021" Jun 06 22:07:13.372291 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.372264 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:13.426819 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.426766 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:13.576517 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.576411 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:13.744703 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.744670 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:13.801643 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.801609 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:13.934747 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:13.934716 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:14.146397 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:14.146365 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:14.339645 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:14.339569 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:14.440609 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:14.440573 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:14.585984 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:14.585956 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:14.642293 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:14.642262 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:14.744600 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:14.744568 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:14.949567 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:14.949478 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:15.105187 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:15.105156 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:15.768393 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:15.768350 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:15.816722 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:15.816691 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:18.106169 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:18.106128 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:18.160671 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:18.160640 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:20.508009 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:20.507980 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:20.508403 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:20.508201 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:25.382165 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:25.382126 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:25.480860 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:25.480829 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:28.372057 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:28.372023 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:28.425458 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:28.425411 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:28.527780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:28.527744 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:30.392118 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:30.392083 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:30.447988 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:30.447956 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:32.571796 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:32.571764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:32.643911 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:32.643879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:34.963347 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:07:34.963284 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ld6pc" podUID="89a055b7-3e1a-4f23-85cf-e60006d08741" Jun 06 22:07:35.438066 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:35.435488 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld6pc" Jun 06 22:07:38.292795 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:38.292758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:07:38.295215 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:38.295178 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/89a055b7-3e1a-4f23-85cf-e60006d08741-metrics-tls\") pod \"dns-default-ld6pc\" (UID: \"89a055b7-3e1a-4f23-85cf-e60006d08741\") " pod="openshift-dns/dns-default-ld6pc" Jun 06 22:07:38.439621 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:38.439594 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j597l\"" Jun 06 22:07:38.446945 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:38.446912 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld6pc" Jun 06 22:07:38.448389 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:38.448360 2580 generic.go:356] "Generic (PLEG): container finished" podID="0444ca20-8bd0-4be6-a4ff-a4afef1f020e" containerID="3ed95d2a522f083a399b939e97530efdb00e1b511f6b602786c1e3878f6bef18" exitCode=0 Jun 06 22:07:38.448496 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:38.448433 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" event={"ID":"0444ca20-8bd0-4be6-a4ff-a4afef1f020e","Type":"ContainerDied","Data":"3ed95d2a522f083a399b939e97530efdb00e1b511f6b602786c1e3878f6bef18"} Jun 06 22:07:38.448798 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:38.448782 2580 scope.go:117] "RemoveContainer" containerID="3ed95d2a522f083a399b939e97530efdb00e1b511f6b602786c1e3878f6bef18" Jun 06 22:07:38.607796 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:38.607769 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ld6pc"] Jun 06 22:07:38.610200 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:07:38.610157 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a055b7_3e1a_4f23_85cf_e60006d08741.slice/crio-2efc53a1a0330891a7d2936047dbfb518d661a82dc6da88240a5abe21e123f22 WatchSource:0}: Error finding container 2efc53a1a0330891a7d2936047dbfb518d661a82dc6da88240a5abe21e123f22: Status 404 returned error can't find the container with id 2efc53a1a0330891a7d2936047dbfb518d661a82dc6da88240a5abe21e123f22 Jun 06 22:07:39.455525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:39.455489 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-vsd4v" event={"ID":"0444ca20-8bd0-4be6-a4ff-a4afef1f020e","Type":"ContainerStarted","Data":"bcac18698ff8a5313e1fafb3fc5ce309427e3e735555bbbea3df48e4753e0eb0"} Jun 06 22:07:39.457237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:39.457211 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ld6pc" event={"ID":"89a055b7-3e1a-4f23-85cf-e60006d08741","Type":"ContainerStarted","Data":"2efc53a1a0330891a7d2936047dbfb518d661a82dc6da88240a5abe21e123f22"} Jun 06 22:07:40.514568 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:40.514545 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:40.537829 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:40.537762 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5fc4968dbb-jzvwh" Jun 06 22:07:41.467749 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:41.467700 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ld6pc" event={"ID":"89a055b7-3e1a-4f23-85cf-e60006d08741","Type":"ContainerStarted","Data":"6d0420609bb8029763ca36f4dda2b04da1e9782f0bc6befcea6f473b6d07ff62"} Jun 06 22:07:41.467957 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:41.467831 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ld6pc" event={"ID":"89a055b7-3e1a-4f23-85cf-e60006d08741","Type":"ContainerStarted","Data":"ab9bbbb06ae3b569e6c887a996e9ce01dc8ea984c5ae7a5213560fe92280fb6a"} Jun 06 22:07:41.467957 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:41.467865 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ld6pc" Jun 06 22:07:41.484078 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:41.484030 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ld6pc" podStartSLOduration=251.584624108 podStartE2EDuration="4m13.484014052s" totalImagesPullingTime="1.899389944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:07:41.483184503 +0000 UTC m=+288.482163989" watchObservedRunningTime="2026-06-06 22:07:41.484014052 +0000 UTC m=+288.482993603" Jun 06 22:07:43.173061 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:43.173026 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:43.222491 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:43.222464 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:51.473565 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:51.473535 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ld6pc" Jun 06 22:07:52.716351 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:52.716302 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:52.815252 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:52.815222 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:53.435282 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:53.435240 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:07:53.435858 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:53.435837 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:07:53.442164 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:53.442140 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:07:53.442615 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:53.442594 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:07:53.446540 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:53.446522 2580 kubelet.go:1648] "Image garbage collection succeeded" Jun 06 22:07:55.504602 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:55.504572 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:55.550146 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:55.550116 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:55.845558 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:55.845475 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:55.892938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:55.892909 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:55.946597 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:55.946570 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:56.201252 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:56.201209 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:56.253443 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:56.253413 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:56.860162 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:56.860129 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-dzfv6_40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764/kube-state-metrics/0.log" Jun 06 22:07:57.059314 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:57.059285 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-dzfv6_40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764/kube-rbac-proxy-main/0.log" Jun 06 22:07:57.260095 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:57.260068 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-dzfv6_40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764/kube-rbac-proxy-self/0.log" Jun 06 22:07:57.459660 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:57.459636 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5fc4968dbb-jzvwh_6096f400-3aba-4410-bf60-7ae2b9bdb31e/metrics-server/0.log" Jun 06 22:07:57.658992 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:57.658970 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-8pfnc_06e6a9e9-c439-423f-98da-80a7e0f105ce/monitoring-plugin/0.log" Jun 06 22:07:58.463305 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:58.463278 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbrjj_2294bbdf-04ad-4961-8d58-01f80a5464f2/init-textfile/0.log" Jun 06 22:07:58.661296 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:58.661267 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbrjj_2294bbdf-04ad-4961-8d58-01f80a5464f2/node-exporter/0.log" Jun 06 22:07:58.860218 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:58.860153 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbrjj_2294bbdf-04ad-4961-8d58-01f80a5464f2/kube-rbac-proxy/0.log" Jun 06 22:07:59.447484 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:59.447438 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:07:59.547614 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:07:59.547579 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:00.258928 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:00.258896 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef28ce25-1474-4edd-99ed-154f8efa1c0d/init-config-reloader/0.log" Jun 06 22:08:00.460755 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:00.460726 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef28ce25-1474-4edd-99ed-154f8efa1c0d/prometheus/0.log" Jun 06 22:08:00.660159 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:00.660136 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef28ce25-1474-4edd-99ed-154f8efa1c0d/config-reloader/0.log" Jun 06 22:08:00.859783 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:00.859752 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef28ce25-1474-4edd-99ed-154f8efa1c0d/thanos-sidecar/0.log" Jun 06 22:08:01.059049 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:01.058982 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef28ce25-1474-4edd-99ed-154f8efa1c0d/kube-rbac-proxy-web/0.log" Jun 06 22:08:01.259699 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:01.259671 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef28ce25-1474-4edd-99ed-154f8efa1c0d/kube-rbac-proxy/0.log" Jun 06 22:08:01.460131 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:01.460105 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef28ce25-1474-4edd-99ed-154f8efa1c0d/kube-rbac-proxy-thanos/0.log" Jun 06 22:08:01.660710 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:01.660682 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-5mdqd_b815c5c4-8fee-4374-a16f-8bf93b94d346/prometheus-operator/0.log" Jun 06 22:08:01.860149 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:01.860074 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-5mdqd_b815c5c4-8fee-4374-a16f-8bf93b94d346/kube-rbac-proxy/0.log" Jun 06 22:08:02.058633 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:02.058599 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-rvvb6_819aa35f-218e-4f27-b32f-6d3d4adc9e26/prometheus-operator-admission-webhook/0.log" Jun 06 22:08:02.263720 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:02.263690 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c97f86fb6-jpmvc_79a9d2e2-54a3-4dca-b503-0a8110954c69/telemeter-client/0.log" Jun 06 22:08:02.401715 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:02.401682 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:02.417297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:02.417274 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:02.459581 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:02.459554 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c97f86fb6-jpmvc_79a9d2e2-54a3-4dca-b503-0a8110954c69/reload/0.log" Jun 06 22:08:02.570146 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:02.570074 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:02.659798 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:02.659773 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c97f86fb6-jpmvc_79a9d2e2-54a3-4dca-b503-0a8110954c69/kube-rbac-proxy/0.log" Jun 06 22:08:04.059681 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:04.059654 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-hhv9q_ff32d99a-8480-4abf-91d3-8eb414d4bf85/networking-console-plugin/0.log" Jun 06 22:08:04.259704 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:04.259674 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:08:04.461513 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:04.461470 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/3.log" Jun 06 22:08:04.577888 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:04.577857 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:04.624946 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:04.624919 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:05.658869 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:05.658842 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ld6pc_89a055b7-3e1a-4f23-85cf-e60006d08741/dns/0.log" Jun 06 22:08:05.860932 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:05.860907 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ld6pc_89a055b7-3e1a-4f23-85cf-e60006d08741/kube-rbac-proxy/0.log" Jun 06 22:08:06.661944 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:06.661917 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bp8js_c6c24d13-14a9-4f94-8112-0ff9d1f62946/dns-node-resolver/0.log" Jun 06 22:08:07.860206 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:07.860179 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-txkmb_9862602e-db24-4b4a-ab24-a6b16129404e/node-ca/0.log" Jun 06 22:08:08.259336 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:08.259309 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78476cbbd5-ldsl9_17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d/router/0.log" Jun 06 22:08:08.414929 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:08.414895 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:08.464881 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:08.464852 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:08.659008 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:08.658983 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n5r5h_51877535-3dba-451c-912e-38b3b31cab69/serve-healthcheck-canary/0.log" Jun 06 22:08:08.804656 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:08.804625 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:08.907038 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:08.907007 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:09.007830 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:09.007713 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:11.947906 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:11.947876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:12.048190 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:12.048156 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:13.518646 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:13.518615 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:13.618206 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:13.618170 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:15.389503 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:15.389470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:15.444957 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:15.444926 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:16.780700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:16.780665 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:16.883682 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:16.883651 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:16.948188 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:16.948157 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:20.444003 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.443962 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 06 22:08:20.444619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.444346 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="prometheus" containerID="cri-o://4084830a8108c96f99f015e796118733d7e11b9069f1233d9b090a3816e1ad5e" gracePeriod=600 Jun 06 22:08:20.444781 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.444356 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy" containerID="cri-o://64aeb10741188deb79e15f8234545dfa9dfdbdaa95bf7b5a8d0b7de31aadc2ca" gracePeriod=600 Jun 06 22:08:20.444858 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.444387 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e751aa900e5627d7477e880a0b52941028512555a95e04e4469e03ccdf049626" gracePeriod=600 Jun 06 22:08:20.444946 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.444414 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="config-reloader" containerID="cri-o://4c7bf733d77ca7a880277d8fa0723f5f278c67afc4ad098eff3d3df7aef5825b" gracePeriod=600 Jun 06 22:08:20.445083 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.444409 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy-web" containerID="cri-o://de597d9b65b481ba528e597b52c29fe4da68c5bff6f3bdaea51b1ce7bcf777dd" gracePeriod=600 Jun 06 22:08:20.445083 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.444426 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="thanos-sidecar" containerID="cri-o://48b2d0644a9e5ed59ee5727fde91caf20a2ce423dc69de98d9fef8d597a1e5a4" gracePeriod=600 Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634050 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerID="e751aa900e5627d7477e880a0b52941028512555a95e04e4469e03ccdf049626" exitCode=0 Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634083 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerID="64aeb10741188deb79e15f8234545dfa9dfdbdaa95bf7b5a8d0b7de31aadc2ca" exitCode=0 Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634092 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerID="48b2d0644a9e5ed59ee5727fde91caf20a2ce423dc69de98d9fef8d597a1e5a4" exitCode=0 Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634101 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerID="4c7bf733d77ca7a880277d8fa0723f5f278c67afc4ad098eff3d3df7aef5825b" exitCode=0 Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634109 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerID="4084830a8108c96f99f015e796118733d7e11b9069f1233d9b090a3816e1ad5e" exitCode=0 Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerDied","Data":"e751aa900e5627d7477e880a0b52941028512555a95e04e4469e03ccdf049626"} Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634222 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerDied","Data":"64aeb10741188deb79e15f8234545dfa9dfdbdaa95bf7b5a8d0b7de31aadc2ca"} Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634265 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerDied","Data":"48b2d0644a9e5ed59ee5727fde91caf20a2ce423dc69de98d9fef8d597a1e5a4"} Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634304 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerDied","Data":"4c7bf733d77ca7a880277d8fa0723f5f278c67afc4ad098eff3d3df7aef5825b"} Jun 06 22:08:20.634395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:20.634345 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerDied","Data":"4084830a8108c96f99f015e796118733d7e11b9069f1233d9b090a3816e1ad5e"} Jun 06 22:08:21.644195 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.644163 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerID="de597d9b65b481ba528e597b52c29fe4da68c5bff6f3bdaea51b1ce7bcf777dd" exitCode=0 Jun 06 22:08:21.644536 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.644230 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerDied","Data":"de597d9b65b481ba528e597b52c29fe4da68c5bff6f3bdaea51b1ce7bcf777dd"} Jun 06 22:08:21.697817 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.697779 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:21.759367 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759333 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-grpc-tls\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759522 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759384 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-db\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759522 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759413 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-thanos-prometheus-http-client-file\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759522 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759452 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-2\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759522 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759485 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-tls-assets\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759744 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759605 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-metrics-client-certs\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759744 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759643 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-trusted-ca-bundle\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759744 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759677 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-kubelet-serving-ca-bundle\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759744 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759710 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-kube-rbac-proxy\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759744 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759740 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-0\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759946 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759765 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-tls\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759946 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759834 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759946 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759873 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759946 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759904 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-serving-certs-ca-bundle\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.759946 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759932 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-1\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.760118 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759958 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config-out\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.760118 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.759983 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-metrics-client-ca\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.760118 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.760012 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddg4n\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-kube-api-access-ddg4n\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.760118 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.760061 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-web-config\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.760118 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.760094 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config\") pod \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\" (UID: \"ef28ce25-1474-4edd-99ed-154f8efa1c0d\") " Jun 06 22:08:21.761555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.760466 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.761555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.760955 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.761555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.761060 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.761555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.761341 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.761555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.761506 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.762948 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.762635 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.762948 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.762729 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:08:21.763980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.763674 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:08:21.764482 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.764447 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:08:21.764672 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.764616 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.765372 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.765338 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.766818 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.766693 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.766818 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.766761 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.766818 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.766774 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.767010 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.766923 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.767063 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.767008 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-kube-api-access-ddg4n" (OuterVolumeSpecName: "kube-api-access-ddg4n") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "kube-api-access-ddg4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:08:21.767113 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.767074 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.767277 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.767246 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config-out" (OuterVolumeSpecName: "config-out") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:08:21.767362 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.767300 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config" (OuterVolumeSpecName: "config") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.779708 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.779681 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-web-config" (OuterVolumeSpecName: "web-config") pod "ef28ce25-1474-4edd-99ed-154f8efa1c0d" (UID: "ef28ce25-1474-4edd-99ed-154f8efa1c0d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:08:21.860938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860867 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-db\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.860938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860894 2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.860938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860905 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.860938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860913 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-tls-assets\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.860938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860922 2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-metrics-client-certs\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.860938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860931 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.860938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860940 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860948 2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-kube-rbac-proxy\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860958 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860966 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860974 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860983 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.860992 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.861000 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.861009 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config-out\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.861016 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef28ce25-1474-4edd-99ed-154f8efa1c0d-configmap-metrics-client-ca\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.861024 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddg4n\" (UniqueName: \"kubernetes.io/projected/ef28ce25-1474-4edd-99ed-154f8efa1c0d-kube-api-access-ddg4n\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.861033 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-web-config\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.861041 2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-config\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:21.861222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:21.861048 2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef28ce25-1474-4edd-99ed-154f8efa1c0d-secret-grpc-tls\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:08:22.651174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.651146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef28ce25-1474-4edd-99ed-154f8efa1c0d","Type":"ContainerDied","Data":"ff8ed26e6449f370cdb0f20d8931aff62f4f9defdf6e5335f7d5107888c51ab9"} Jun 06 22:08:22.651655 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.651206 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.651655 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.651217 2580 scope.go:117] "RemoveContainer" containerID="e751aa900e5627d7477e880a0b52941028512555a95e04e4469e03ccdf049626" Jun 06 22:08:22.659089 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.658918 2580 scope.go:117] "RemoveContainer" containerID="64aeb10741188deb79e15f8234545dfa9dfdbdaa95bf7b5a8d0b7de31aadc2ca" Jun 06 22:08:22.667872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.667855 2580 scope.go:117] "RemoveContainer" containerID="de597d9b65b481ba528e597b52c29fe4da68c5bff6f3bdaea51b1ce7bcf777dd" Jun 06 22:08:22.678928 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.677798 2580 scope.go:117] "RemoveContainer" containerID="48b2d0644a9e5ed59ee5727fde91caf20a2ce423dc69de98d9fef8d597a1e5a4" Jun 06 22:08:22.680173 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.679234 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 06 22:08:22.681649 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.681625 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 06 22:08:22.687673 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.687654 2580 scope.go:117] "RemoveContainer" containerID="4c7bf733d77ca7a880277d8fa0723f5f278c67afc4ad098eff3d3df7aef5825b" Jun 06 22:08:22.694781 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.694760 2580 scope.go:117] "RemoveContainer" containerID="4084830a8108c96f99f015e796118733d7e11b9069f1233d9b090a3816e1ad5e" Jun 06 22:08:22.701994 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.701972 2580 scope.go:117] "RemoveContainer" containerID="a98106c357295708d33a1b309616a3e47877ebdad334e7d2eceadd180fdf131c" Jun 06 22:08:22.703432 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703411 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 06 22:08:22.703707 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703694 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="init-config-reloader" Jun 06 22:08:22.703707 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703706 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="init-config-reloader" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703714 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="prometheus" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703720 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="prometheus" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703728 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy-thanos" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703734 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy-thanos" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703741 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703746 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703753 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="config-reloader" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703758 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="config-reloader" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703769 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="thanos-sidecar" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703774 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="thanos-sidecar" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703787 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy-web" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703792 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy-web" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703863 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="prometheus" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703877 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy-thanos" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703888 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="config-reloader" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703896 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy-web" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703905 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="thanos-sidecar" Jun 06 22:08:22.704163 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.703913 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" containerName="kube-rbac-proxy" Jun 06 22:08:22.708958 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.708941 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.711879 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.711843 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 06 22:08:22.712484 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712289 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 06 22:08:22.712484 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712375 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 06 22:08:22.712484 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712463 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 06 22:08:22.712484 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712483 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-co2uctam9c9oc\"" Jun 06 22:08:22.712781 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 06 22:08:22.712781 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712689 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 06 22:08:22.712781 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712704 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b896k\"" Jun 06 22:08:22.712781 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712744 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 06 22:08:22.713189 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712799 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 06 22:08:22.713189 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.712868 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 06 22:08:22.713713 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.713693 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 06 22:08:22.713848 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.713702 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 06 22:08:22.713941 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.713877 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 06 22:08:22.715925 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.715905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 06 22:08:22.717224 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.717207 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 06 22:08:22.721670 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.721649 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 06 22:08:22.768281 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768253 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4e87b7d-bf13-4db9-8872-32f07f762d8d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768281 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768281 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768487 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768300 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-web-config\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768487 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768316 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768487 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768391 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768487 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768487 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768487 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrl96\" (UniqueName: \"kubernetes.io/projected/a4e87b7d-bf13-4db9-8872-32f07f762d8d-kube-api-access-jrl96\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768487 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4e87b7d-bf13-4db9-8872-32f07f762d8d-config-out\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768696 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768761 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768790 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-config\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.768872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768857 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.769197 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.768888 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870302 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-web-config\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870302 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870302 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870283 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870520 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870520 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870520 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrl96\" (UniqueName: \"kubernetes.io/projected/a4e87b7d-bf13-4db9-8872-32f07f762d8d-kube-api-access-jrl96\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870668 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870578 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870668 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870770 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870770 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870693 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870770 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870968 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4e87b7d-bf13-4db9-8872-32f07f762d8d-config-out\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870968 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.870968 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870897 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.871711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.871182 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.871711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.871193 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.871711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.870932 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.871711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.871256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-config\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.871711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.871314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.871711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.871352 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.871711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.871380 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4e87b7d-bf13-4db9-8872-32f07f762d8d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.871711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.871658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.872228 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.871977 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.872336 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.873163 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.873379 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.873791 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-web-config\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.874323 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.874713 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-config\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.875110 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.875518 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.875577 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.875764 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4e87b7d-bf13-4db9-8872-32f07f762d8d-config-out\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.875827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.875766 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.876467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.875861 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.876467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.876218 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a4e87b7d-bf13-4db9-8872-32f07f762d8d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.876467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.876269 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.876629 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.876463 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a4e87b7d-bf13-4db9-8872-32f07f762d8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.876629 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.876590 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4e87b7d-bf13-4db9-8872-32f07f762d8d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:22.878223 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:22.878206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrl96\" (UniqueName: \"kubernetes.io/projected/a4e87b7d-bf13-4db9-8872-32f07f762d8d-kube-api-access-jrl96\") pod \"prometheus-k8s-0\" (UID: \"a4e87b7d-bf13-4db9-8872-32f07f762d8d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:23.020721 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:23.020668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:23.157525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:23.157500 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 06 22:08:23.159789 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:08:23.159764 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e87b7d_bf13_4db9_8872_32f07f762d8d.slice/crio-7a60f226aa82360e853910ef84a570295ad8b8f369e06d525d4a018fb02dcfa9 WatchSource:0}: Error finding container 7a60f226aa82360e853910ef84a570295ad8b8f369e06d525d4a018fb02dcfa9: Status 404 returned error can't find the container with id 7a60f226aa82360e853910ef84a570295ad8b8f369e06d525d4a018fb02dcfa9 Jun 06 22:08:23.487858 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:23.487771 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef28ce25-1474-4edd-99ed-154f8efa1c0d" path="/var/lib/kubelet/pods/ef28ce25-1474-4edd-99ed-154f8efa1c0d/volumes" Jun 06 22:08:23.656521 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:23.656492 2580 generic.go:356] "Generic (PLEG): container finished" podID="a4e87b7d-bf13-4db9-8872-32f07f762d8d" containerID="2959be49a336f59aecb7941b3cc17b9752a610c26eb1f801ce6dcd13004e52d2" exitCode=0 Jun 06 22:08:23.656950 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:23.656579 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4e87b7d-bf13-4db9-8872-32f07f762d8d","Type":"ContainerDied","Data":"2959be49a336f59aecb7941b3cc17b9752a610c26eb1f801ce6dcd13004e52d2"} Jun 06 22:08:23.656950 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:23.656644 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4e87b7d-bf13-4db9-8872-32f07f762d8d","Type":"ContainerStarted","Data":"7a60f226aa82360e853910ef84a570295ad8b8f369e06d525d4a018fb02dcfa9"} Jun 06 22:08:24.074483 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.074456 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:24.196153 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.196118 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:24.264359 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.264291 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:24.606927 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.606852 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:24.654582 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.654555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:24.665072 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.665045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4e87b7d-bf13-4db9-8872-32f07f762d8d","Type":"ContainerStarted","Data":"33b2130e6c93721fa0503fc084d6a5e4ecbe51c896f1a3ff8b5b1b7918a19576"} Jun 06 22:08:24.665412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.665133 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4e87b7d-bf13-4db9-8872-32f07f762d8d","Type":"ContainerStarted","Data":"52774a8fed50311f645f6b2e860e781512651cb21444c558b2ce0c33ebd5b54c"} Jun 06 22:08:24.665412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.665176 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4e87b7d-bf13-4db9-8872-32f07f762d8d","Type":"ContainerStarted","Data":"24907855581bd32fa79225462c55b9aa83ff55bd19eb2da24601d85fd90baad4"} Jun 06 22:08:24.665412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.665204 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4e87b7d-bf13-4db9-8872-32f07f762d8d","Type":"ContainerStarted","Data":"1dba12f83122e3be8c29eec7daaa46af84173f7b4a872a1fe6e8f24db8987c5d"} Jun 06 22:08:24.665412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.665232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4e87b7d-bf13-4db9-8872-32f07f762d8d","Type":"ContainerStarted","Data":"907ccc3c4ec8766c660bd4b482dbb749ebef62f504aef8dcd80a589b4d1f0330"} Jun 06 22:08:24.665412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.665281 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4e87b7d-bf13-4db9-8872-32f07f762d8d","Type":"ContainerStarted","Data":"28ff0ebfad4b90067aef79cac811352adf9456ee41894bb2c0f7ba3b91405169"} Jun 06 22:08:24.692876 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:24.692827 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.674444933 podStartE2EDuration="2.692795266s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.350333ms" isStatefulPod=false podCreationTimestamp="2026-06-06 22:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:08:24.690123469 +0000 UTC m=+331.689102979" watchObservedRunningTime="2026-06-06 22:08:24.692795266 +0000 UTC m=+331.691774751" Jun 06 22:08:28.021648 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:28.021600 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:08:30.400711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:30.400679 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:08:30.502380 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:08:30.502346 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:21.224954 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:21.224920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:21.318542 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:21.318511 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:23.021596 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:23.021541 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:09:23.037173 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:23.037148 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:09:23.236795 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:23.236765 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:23.284798 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:23.284716 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:23.914218 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:23.914185 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 06 22:09:47.540408 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:47.540374 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:48.916985 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:48.916951 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:51.140870 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:51.140826 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:51.243923 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:51.243889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:52.020278 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:52.020243 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:52.119473 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:52.119438 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:52.561088 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:52.561051 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:52.688570 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:52.688537 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:55.331768 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:55.331733 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:55.908308 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:55.908274 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:57.943518 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:57.943485 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:09:58.044311 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:09:58.044285 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:00.624463 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:00.624426 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:00.672905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:00.672876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:03.914933 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:03.914900 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:05.429499 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:05.429466 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:05.529815 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:05.529763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:07.135469 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.135435 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv"] Jun 06 22:10:07.138724 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.138707 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" Jun 06 22:10:07.141193 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.141166 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-t8lhj\"" Jun 06 22:10:07.141314 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.141234 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 06 22:10:07.142386 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.142363 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 06 22:10:07.146851 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.146829 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv"] Jun 06 22:10:07.254500 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.254463 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tshcj\" (UniqueName: \"kubernetes.io/projected/a0553f51-0f78-416a-911d-cc086d650427-kube-api-access-tshcj\") pod \"openshift-lws-operator-fd8ccff4c-f8rcv\" (UID: \"a0553f51-0f78-416a-911d-cc086d650427\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" Jun 06 22:10:07.254671 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.254517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0553f51-0f78-416a-911d-cc086d650427-tmp\") pod \"openshift-lws-operator-fd8ccff4c-f8rcv\" (UID: \"a0553f51-0f78-416a-911d-cc086d650427\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" Jun 06 22:10:07.355257 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.355222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tshcj\" (UniqueName: \"kubernetes.io/projected/a0553f51-0f78-416a-911d-cc086d650427-kube-api-access-tshcj\") pod \"openshift-lws-operator-fd8ccff4c-f8rcv\" (UID: \"a0553f51-0f78-416a-911d-cc086d650427\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" Jun 06 22:10:07.355418 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.355276 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0553f51-0f78-416a-911d-cc086d650427-tmp\") pod \"openshift-lws-operator-fd8ccff4c-f8rcv\" (UID: \"a0553f51-0f78-416a-911d-cc086d650427\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" Jun 06 22:10:07.355630 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.355616 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0553f51-0f78-416a-911d-cc086d650427-tmp\") pod \"openshift-lws-operator-fd8ccff4c-f8rcv\" (UID: \"a0553f51-0f78-416a-911d-cc086d650427\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" Jun 06 22:10:07.368202 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.368174 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tshcj\" (UniqueName: \"kubernetes.io/projected/a0553f51-0f78-416a-911d-cc086d650427-kube-api-access-tshcj\") pod \"openshift-lws-operator-fd8ccff4c-f8rcv\" (UID: \"a0553f51-0f78-416a-911d-cc086d650427\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" Jun 06 22:10:07.462555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.462523 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" Jun 06 22:10:07.595834 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.595783 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv"] Jun 06 22:10:07.598993 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:10:07.598958 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0553f51_0f78_416a_911d_cc086d650427.slice/crio-bb9723661c297aa353e61eac21b357f8982e62bc36ecf53b5290b8b5d7d1a3cb WatchSource:0}: Error finding container bb9723661c297aa353e61eac21b357f8982e62bc36ecf53b5290b8b5d7d1a3cb: Status 404 returned error can't find the container with id bb9723661c297aa353e61eac21b357f8982e62bc36ecf53b5290b8b5d7d1a3cb Jun 06 22:10:07.600475 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:07.600459 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 06 22:10:08.072203 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:08.072171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" event={"ID":"a0553f51-0f78-416a-911d-cc086d650427","Type":"ContainerStarted","Data":"bb9723661c297aa353e61eac21b357f8982e62bc36ecf53b5290b8b5d7d1a3cb"} Jun 06 22:10:11.087698 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:11.087606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" event={"ID":"a0553f51-0f78-416a-911d-cc086d650427","Type":"ContainerStarted","Data":"f22f0371010b1cfff640d9fc109f71d69cd6d9b715d540e87c4d3412a86f6b37"} Jun 06 22:10:11.104945 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:11.104895 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-f8rcv" podStartSLOduration=0.896508485 podStartE2EDuration="4.104879415s" totalImagesPullingTime="3.20837093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:10:11.103141847 +0000 UTC m=+438.102121360" watchObservedRunningTime="2026-06-06 22:10:11.104879415 +0000 UTC m=+438.103858899" Jun 06 22:10:11.792210 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:11.792174 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:11.893038 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:11.893008 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:14.391499 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:14.391423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:26.386430 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:26.386396 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:26.483404 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:26.483352 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:26.985212 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:26.985183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:27.198674 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.198638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:27.283630 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.283563 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:27.316556 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.316524 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8"] Jun 06 22:10:27.319932 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.319910 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.323328 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.323305 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 06 22:10:27.323639 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.323582 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 06 22:10:27.323762 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.323641 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 06 22:10:27.323848 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.323829 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dbwnd\"" Jun 06 22:10:27.324086 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.324069 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 06 22:10:27.337695 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.337673 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8"] Jun 06 22:10:27.428301 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.428270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5e0e609d-72e2-4d57-9139-48dc0560fdcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.428662 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.428320 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0e609d-72e2-4d57-9139-48dc0560fdcf-tmp\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.428662 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.428344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5e0e609d-72e2-4d57-9139-48dc0560fdcf-manifests\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.428662 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.428425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e0e609d-72e2-4d57-9139-48dc0560fdcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.428662 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.428467 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rh6d\" (UniqueName: \"kubernetes.io/projected/5e0e609d-72e2-4d57-9139-48dc0560fdcf-kube-api-access-8rh6d\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.491240 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.491209 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:27.529364 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.529339 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0e609d-72e2-4d57-9139-48dc0560fdcf-tmp\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.529464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.529372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5e0e609d-72e2-4d57-9139-48dc0560fdcf-manifests\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.529464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.529403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e0e609d-72e2-4d57-9139-48dc0560fdcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.529464 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.529425 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rh6d\" (UniqueName: \"kubernetes.io/projected/5e0e609d-72e2-4d57-9139-48dc0560fdcf-kube-api-access-8rh6d\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.529565 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.529468 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5e0e609d-72e2-4d57-9139-48dc0560fdcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.529841 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.529790 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5e0e609d-72e2-4d57-9139-48dc0560fdcf-manifests\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.531592 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.531567 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0e609d-72e2-4d57-9139-48dc0560fdcf-tmp\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.531926 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.531908 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5e0e609d-72e2-4d57-9139-48dc0560fdcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.531981 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.531935 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e0e609d-72e2-4d57-9139-48dc0560fdcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.537506 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.537462 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rh6d\" (UniqueName: \"kubernetes.io/projected/5e0e609d-72e2-4d57-9139-48dc0560fdcf-kube-api-access-8rh6d\") pod \"opendatahub-operator-controller-manager-9c76bcc58-lk6p8\" (UID: \"5e0e609d-72e2-4d57-9139-48dc0560fdcf\") " pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.592313 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.592274 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:27.630320 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.630287 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:27.692515 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.692461 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:27.772065 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.772027 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:27.773473 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.773452 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8"] Jun 06 22:10:27.776273 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:10:27.776246 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0e609d_72e2_4d57_9139_48dc0560fdcf.slice/crio-4304cf7de1c240017029d6a86736d569962e44d69621430e230793df3adc6c08 WatchSource:0}: Error finding container 4304cf7de1c240017029d6a86736d569962e44d69621430e230793df3adc6c08: Status 404 returned error can't find the container with id 4304cf7de1c240017029d6a86736d569962e44d69621430e230793df3adc6c08 Jun 06 22:10:27.832694 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.832618 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:27.935692 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:27.935655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:28.035436 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:28.035403 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:28.161000 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:28.160965 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" event={"ID":"5e0e609d-72e2-4d57-9139-48dc0560fdcf","Type":"ContainerStarted","Data":"4304cf7de1c240017029d6a86736d569962e44d69621430e230793df3adc6c08"} Jun 06 22:10:31.176872 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:31.176833 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" event={"ID":"5e0e609d-72e2-4d57-9139-48dc0560fdcf","Type":"ContainerStarted","Data":"fc2f9fa1c24cf60552bb996c448bfa874ba8f9ae97ad5a68e33eb88fdf21e9d6"} Jun 06 22:10:32.182934 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:32.182904 2580 generic.go:356] "Generic (PLEG): container finished" podID="5e0e609d-72e2-4d57-9139-48dc0560fdcf" containerID="fc2f9fa1c24cf60552bb996c448bfa874ba8f9ae97ad5a68e33eb88fdf21e9d6" exitCode=0 Jun 06 22:10:32.183327 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:32.182994 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" event={"ID":"5e0e609d-72e2-4d57-9139-48dc0560fdcf","Type":"ContainerDied","Data":"fc2f9fa1c24cf60552bb996c448bfa874ba8f9ae97ad5a68e33eb88fdf21e9d6"} Jun 06 22:10:33.189138 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:33.189097 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" event={"ID":"5e0e609d-72e2-4d57-9139-48dc0560fdcf","Type":"ContainerStarted","Data":"a65edb471f1ed396783b9953218c936521f42bac33149a390c7b877e2c51814b"} Jun 06 22:10:33.189542 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:33.189226 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:33.215184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:33.215138 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" podStartSLOduration=2.031400217 podStartE2EDuration="6.215123319s" totalImagesPullingTime="3.276313234s" totalInitContainerRuntime="907.409868ms" isStatefulPod=false podCreationTimestamp="2026-06-06 22:10:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:10:33.21345235 +0000 UTC m=+460.212431835" watchObservedRunningTime="2026-06-06 22:10:33.215123319 +0000 UTC m=+460.214102807" Jun 06 22:10:38.582726 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:38.582695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:38.682450 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:38.682423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:44.196337 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:44.196306 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9c76bcc58-lk6p8" Jun 06 22:10:46.107575 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.107521 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:46.291824 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.291772 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:46.402392 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.402363 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:46.504280 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.504246 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:46.603099 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.603061 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:46.707301 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.707212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:46.813370 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.813341 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:46.885572 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.885545 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:46.941850 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.941820 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv"] Jun 06 22:10:46.955542 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.955512 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:46.955711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.955582 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv"] Jun 06 22:10:46.958323 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.958269 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 06 22:10:46.959657 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.959638 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 06 22:10:46.959765 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.959638 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 06 22:10:46.959765 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.959717 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-l92mj\"" Jun 06 22:10:46.966646 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.966625 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:46.997995 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.997959 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee3bce4f-38ba-4bdb-855d-a563408762ff-metrics-cert\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:46.998143 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.998000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee3bce4f-38ba-4bdb-855d-a563408762ff-cert\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:46.998143 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.998018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kvtc\" (UniqueName: \"kubernetes.io/projected/ee3bce4f-38ba-4bdb-855d-a563408762ff-kube-api-access-6kvtc\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:46.998257 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:46.998163 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ee3bce4f-38ba-4bdb-855d-a563408762ff-manager-config\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.090756 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.090722 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:47.098874 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.098849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ee3bce4f-38ba-4bdb-855d-a563408762ff-manager-config\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.099004 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.098924 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee3bce4f-38ba-4bdb-855d-a563408762ff-metrics-cert\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.099004 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.098956 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee3bce4f-38ba-4bdb-855d-a563408762ff-cert\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.099004 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.098979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kvtc\" (UniqueName: \"kubernetes.io/projected/ee3bce4f-38ba-4bdb-855d-a563408762ff-kube-api-access-6kvtc\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.099520 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.099492 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ee3bce4f-38ba-4bdb-855d-a563408762ff-manager-config\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.101470 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.101449 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee3bce4f-38ba-4bdb-855d-a563408762ff-cert\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.101582 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.101520 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee3bce4f-38ba-4bdb-855d-a563408762ff-metrics-cert\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.108099 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.108065 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kvtc\" (UniqueName: \"kubernetes.io/projected/ee3bce4f-38ba-4bdb-855d-a563408762ff-kube-api-access-6kvtc\") pod \"lws-controller-manager-7774b9f8d5-4g7zv\" (UID: \"ee3bce4f-38ba-4bdb-855d-a563408762ff\") " pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.267721 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.267627 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:47.310558 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.310526 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:47.414608 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.414578 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv"] Jun 06 22:10:47.418119 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:10:47.418089 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee3bce4f_38ba_4bdb_855d_a563408762ff.slice/crio-e3489dcce291bf0bb282bea992941025feff94681b23fc71e8fda46463bb6b43 WatchSource:0}: Error finding container e3489dcce291bf0bb282bea992941025feff94681b23fc71e8fda46463bb6b43: Status 404 returned error can't find the container with id e3489dcce291bf0bb282bea992941025feff94681b23fc71e8fda46463bb6b43 Jun 06 22:10:47.539359 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:47.539281 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:48.255608 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:48.255570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" event={"ID":"ee3bce4f-38ba-4bdb-855d-a563408762ff","Type":"ContainerStarted","Data":"e3489dcce291bf0bb282bea992941025feff94681b23fc71e8fda46463bb6b43"} Jun 06 22:10:51.270313 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:51.270272 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" event={"ID":"ee3bce4f-38ba-4bdb-855d-a563408762ff","Type":"ContainerStarted","Data":"e2963d7dcf4e0b8b8aeff3e3291b414bbcef3ea159778d9386aed39f94800e15"} Jun 06 22:10:51.270759 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:51.270345 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:10:51.286980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:51.286932 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" podStartSLOduration=1.901615493 podStartE2EDuration="5.286914936s" totalImagesPullingTime="3.385299443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:10:51.285831635 +0000 UTC m=+478.284811119" watchObservedRunningTime="2026-06-06 22:10:51.286914936 +0000 UTC m=+478.285894420" Jun 06 22:10:56.755889 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:56.755853 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:56.848099 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:56.848069 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:57.028111 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:57.028024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:57.340205 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:57.340121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:58.014728 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:58.014695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:58.128444 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:58.128415 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:10:58.201514 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:10:58.201475 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:02.276227 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:02.276190 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7774b9f8d5-4g7zv" Jun 06 22:11:02.605783 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:02.605706 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:02.712863 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:02.712825 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:03.804516 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:03.804463 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:03.905582 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:03.905529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:04.465404 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:04.465368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:04.820647 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:04.820532 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:05.083041 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:05.082959 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:15.388953 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:15.388921 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:15.505397 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:15.505364 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:22.414387 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.414351 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-64ff74d9d5-2jnzz"] Jun 06 22:11:22.418343 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.418321 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" Jun 06 22:11:22.420934 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.420903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgts6\" (UniqueName: \"kubernetes.io/projected/8d9a23a6-63f4-4caf-82b8-4528af3f260c-kube-api-access-vgts6\") pod \"dashboard-redirect-64ff74d9d5-2jnzz\" (UID: \"8d9a23a6-63f4-4caf-82b8-4528af3f260c\") " pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" Jun 06 22:11:22.421046 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.420952 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fml2j\"" Jun 06 22:11:22.421046 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.420950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/8d9a23a6-63f4-4caf-82b8-4528af3f260c-redirect-config\") pod \"dashboard-redirect-64ff74d9d5-2jnzz\" (UID: \"8d9a23a6-63f4-4caf-82b8-4528af3f260c\") " pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" Jun 06 22:11:22.421046 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.420999 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 06 22:11:22.429329 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.429304 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-64ff74d9d5-2jnzz"] Jun 06 22:11:22.432407 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.432383 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:22.522024 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.521992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgts6\" (UniqueName: \"kubernetes.io/projected/8d9a23a6-63f4-4caf-82b8-4528af3f260c-kube-api-access-vgts6\") pod \"dashboard-redirect-64ff74d9d5-2jnzz\" (UID: \"8d9a23a6-63f4-4caf-82b8-4528af3f260c\") " pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" Jun 06 22:11:22.522191 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.522037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/8d9a23a6-63f4-4caf-82b8-4528af3f260c-redirect-config\") pod \"dashboard-redirect-64ff74d9d5-2jnzz\" (UID: \"8d9a23a6-63f4-4caf-82b8-4528af3f260c\") " pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" Jun 06 22:11:22.522680 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.522662 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/8d9a23a6-63f4-4caf-82b8-4528af3f260c-redirect-config\") pod \"dashboard-redirect-64ff74d9d5-2jnzz\" (UID: \"8d9a23a6-63f4-4caf-82b8-4528af3f260c\") " pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" Jun 06 22:11:22.531015 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.530988 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgts6\" (UniqueName: \"kubernetes.io/projected/8d9a23a6-63f4-4caf-82b8-4528af3f260c-kube-api-access-vgts6\") pod \"dashboard-redirect-64ff74d9d5-2jnzz\" (UID: \"8d9a23a6-63f4-4caf-82b8-4528af3f260c\") " pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" Jun 06 22:11:22.730356 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.730278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" Jun 06 22:11:22.856880 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:22.856852 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-64ff74d9d5-2jnzz"] Jun 06 22:11:22.858513 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:11:22.858484 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d9a23a6_63f4_4caf_82b8_4528af3f260c.slice/crio-e1d7280c90d48f4d816f922b86ac10b3217ea40e702c47920550e7c41280f8de WatchSource:0}: Error finding container e1d7280c90d48f4d816f922b86ac10b3217ea40e702c47920550e7c41280f8de: Status 404 returned error can't find the container with id e1d7280c90d48f4d816f922b86ac10b3217ea40e702c47920550e7c41280f8de Jun 06 22:11:23.414766 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:23.414728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" event={"ID":"8d9a23a6-63f4-4caf-82b8-4528af3f260c","Type":"ContainerStarted","Data":"e1d7280c90d48f4d816f922b86ac10b3217ea40e702c47920550e7c41280f8de"} Jun 06 22:11:25.425786 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:25.425750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" event={"ID":"8d9a23a6-63f4-4caf-82b8-4528af3f260c","Type":"ContainerStarted","Data":"c817bbebd3c8a906e82635b59bf0ce1f61dbb89607ce1acdc12fb929b76f7f36"} Jun 06 22:11:25.452879 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:25.452832 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-64ff74d9d5-2jnzz" podStartSLOduration=1.265650667 podStartE2EDuration="3.452817405s" totalImagesPullingTime="2.187166738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:11:25.451541266 +0000 UTC m=+512.450520751" watchObservedRunningTime="2026-06-06 22:11:25.452817405 +0000 UTC m=+512.451796880" Jun 06 22:11:32.060306 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:32.060275 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:32.154702 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:32.154667 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:32.655389 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:32.655357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:35.641066 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.641034 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:35.691434 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.691367 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:35.889481 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.889447 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z"] Jun 06 22:11:35.899888 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.899857 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:35.902532 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.902506 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z"] Jun 06 22:11:35.902654 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.902508 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 06 22:11:35.902723 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.902680 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-p7qj6\"" Jun 06 22:11:35.921999 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.921964 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:35.943320 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.943290 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:35.943427 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.943329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:35.943427 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.943352 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:35.943511 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.943444 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdv8m\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-kube-api-access-xdv8m\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:35.943511 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.943505 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:35.943588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.943527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:35.943588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.943558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:35.943588 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.943573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:35.943691 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:35.943599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044257 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044223 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdv8m\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-kube-api-access-xdv8m\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044423 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044423 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044509 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044509 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044611 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044611 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044553 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044611 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044611 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.044952 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044930 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.045041 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.044981 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.045111 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.045090 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.045156 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.045106 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.045156 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.045111 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.046923 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.046899 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.047142 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.047123 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.052142 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.052121 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.052234 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.052165 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdv8m\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-kube-api-access-xdv8m\") pod \"data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.111283 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.111252 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:36.143608 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.143577 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk"] Jun 06 22:11:36.147724 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.147708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.164544 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.164515 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk"] Jun 06 22:11:36.213379 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.213348 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:36.247440 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.247409 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.247605 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.247456 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.247605 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.247481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.247605 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.247531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.247605 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.247571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f4537ef-f82b-470a-93e4-d1e646796f66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.247605 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.247602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.247797 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.247632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r65jp\" (UniqueName: \"kubernetes.io/projected/2f4537ef-f82b-470a-93e4-d1e646796f66-kube-api-access-r65jp\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.247797 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.247678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.247797 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.247699 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.253073 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.252461 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:36.346383 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.346358 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z"] Jun 06 22:11:36.348342 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348302 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348467 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:11:36.348378 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f4005c_05a8_4d17_ab3c_b5e6b7e7afe4.slice/crio-5ffd4b495a1f9948796b41a413daa4c43cd85b4944619ad89023d3edb20bcb68 WatchSource:0}: Error finding container 5ffd4b495a1f9948796b41a413daa4c43cd85b4944619ad89023d3edb20bcb68: Status 404 returned error can't find the container with id 5ffd4b495a1f9948796b41a413daa4c43cd85b4944619ad89023d3edb20bcb68 Jun 06 22:11:36.348467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348447 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348676 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348676 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348676 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f4537ef-f82b-470a-93e4-d1e646796f66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348676 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348676 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r65jp\" (UniqueName: \"kubernetes.io/projected/2f4537ef-f82b-470a-93e4-d1e646796f66-kube-api-access-r65jp\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348949 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348777 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348949 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348838 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348949 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348864 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.348949 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.348888 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.349525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.349501 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f4537ef-f82b-470a-93e4-d1e646796f66-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.351100 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.351076 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.351426 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.351409 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.359404 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.359381 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r65jp\" (UniqueName: \"kubernetes.io/projected/2f4537ef-f82b-470a-93e4-d1e646796f66-kube-api-access-r65jp\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.359686 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.359671 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4537ef-f82b-470a-93e4-d1e646796f66-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f979554rqk\" (UID: \"2f4537ef-f82b-470a-93e4-d1e646796f66\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.458039 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.457954 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:36.478674 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.478644 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" event={"ID":"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4","Type":"ContainerStarted","Data":"5ffd4b495a1f9948796b41a413daa4c43cd85b4944619ad89023d3edb20bcb68"} Jun 06 22:11:36.588754 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:36.588729 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk"] Jun 06 22:11:36.591094 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:11:36.591065 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4537ef_f82b_470a_93e4_d1e646796f66.slice/crio-680e02c805153d5983c8e6045e76e366bbe25af735fbd8c4812c9475707ae3cc WatchSource:0}: Error finding container 680e02c805153d5983c8e6045e76e366bbe25af735fbd8c4812c9475707ae3cc: Status 404 returned error can't find the container with id 680e02c805153d5983c8e6045e76e366bbe25af735fbd8c4812c9475707ae3cc Jun 06 22:11:37.491420 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:37.491381 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" event={"ID":"2f4537ef-f82b-470a-93e4-d1e646796f66","Type":"ContainerStarted","Data":"680e02c805153d5983c8e6045e76e366bbe25af735fbd8c4812c9475707ae3cc"} Jun 06 22:11:39.036407 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.036366 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:11:39.036700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.036435 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:11:39.036700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.036464 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:11:39.041863 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.041829 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:11:39.041978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.041893 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:11:39.041978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.041922 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:11:39.499597 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.499558 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" event={"ID":"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4","Type":"ContainerStarted","Data":"d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5"} Jun 06 22:11:39.501306 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.501274 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" event={"ID":"2f4537ef-f82b-470a-93e4-d1e646796f66","Type":"ContainerStarted","Data":"3514de08ed2af7e4464619ae7a1012d93e98d3f8922a09f27705918bacc614bb"} Jun 06 22:11:39.521900 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.521855 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" podStartSLOduration=1.836699791 podStartE2EDuration="4.521839934s" totalImagesPullingTime="2.685140143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:11:39.519635429 +0000 UTC m=+526.518614934" watchObservedRunningTime="2026-06-06 22:11:39.521839934 +0000 UTC m=+526.520819417" Jun 06 22:11:39.544206 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.544159 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" podStartSLOduration=1.095665287 podStartE2EDuration="3.544143415s" totalImagesPullingTime="2.448478128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:11:39.542619199 +0000 UTC m=+526.541598683" watchObservedRunningTime="2026-06-06 22:11:39.544143415 +0000 UTC m=+526.543122898" Jun 06 22:11:39.812604 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:39.812523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:40.214173 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:40.214136 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:40.215300 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:40.215273 2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jun 06 22:11:40.215397 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:40.215325 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" podUID="05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jun 06 22:11:40.458800 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:40.458766 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:40.463473 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:40.463451 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:40.506007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:40.505920 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:40.506639 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:40.506621 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f979554rqk" Jun 06 22:11:40.556344 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:40.556310 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z"] Jun 06 22:11:41.214651 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:41.214618 2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jun 06 22:11:41.215055 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:41.214682 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" podUID="05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jun 06 22:11:42.214474 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:42.214442 2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jun 06 22:11:42.214627 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:42.214505 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" podUID="05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jun 06 22:11:42.514369 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:42.514276 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" podUID="05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" containerName="istio-proxy" containerID="cri-o://d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5" gracePeriod=30 Jun 06 22:11:47.661043 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.661004 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:47.775431 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.775375 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:47.856708 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856673 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-certs\") pod \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " Jun 06 22:11:47.856917 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856726 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-envoy\") pod \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " Jun 06 22:11:47.856917 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856747 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istiod-ca-cert\") pod \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " Jun 06 22:11:47.856917 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856777 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-socket\") pod \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " Jun 06 22:11:47.856917 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856813 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-credential-socket\") pod \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " Jun 06 22:11:47.856917 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856854 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-data\") pod \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " Jun 06 22:11:47.856917 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856903 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-token\") pod \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " Jun 06 22:11:47.857250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856957 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdv8m\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-kube-api-access-xdv8m\") pod \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " Jun 06 22:11:47.857250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856993 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-podinfo\") pod \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\" (UID: \"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4\") " Jun 06 22:11:47.857250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.856951 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" (UID: "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:11:47.857250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.857198 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" (UID: "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:11:47.857250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.857051 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" (UID: "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:11:47.857250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.857078 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" (UID: "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:11:47.857250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.857157 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-data" (OuterVolumeSpecName: "istio-data") pod "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" (UID: "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:11:47.857656 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.857357 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istiod-ca-cert\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:11:47.857656 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.857376 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-socket\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:11:47.857656 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.857395 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-credential-socket\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:11:47.857656 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.857416 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-data\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:11:47.857656 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.857429 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-workload-certs\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:11:47.859416 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.859390 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" (UID: "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 06 22:11:47.859542 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.859435 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-token" (OuterVolumeSpecName: "istio-token") pod "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" (UID: "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:11:47.859606 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.859559 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-kube-api-access-xdv8m" (OuterVolumeSpecName: "kube-api-access-xdv8m") pod "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" (UID: "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4"). InnerVolumeSpecName "kube-api-access-xdv8m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:11:47.859606 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.859567 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" (UID: "05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:11:47.866771 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.866750 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mhskb"] Jun 06 22:11:47.867277 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.867255 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" containerName="istio-proxy" Jun 06 22:11:47.867277 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.867278 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" containerName="istio-proxy" Jun 06 22:11:47.867417 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.867368 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" containerName="istio-proxy" Jun 06 22:11:47.871187 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.871159 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mhskb" Jun 06 22:11:47.874073 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.874042 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 06 22:11:47.874308 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.874291 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xx5fs\"" Jun 06 22:11:47.874385 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.874306 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 06 22:11:47.882792 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.882763 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mhskb"] Jun 06 22:11:47.958797 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.958751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpzsc\" (UniqueName: \"kubernetes.io/projected/154a901b-ef4b-48d0-af4c-2458da8b0080-kube-api-access-hpzsc\") pod \"kuadrant-operator-catalog-mhskb\" (UID: \"154a901b-ef4b-48d0-af4c-2458da8b0080\") " pod="kuadrant-system/kuadrant-operator-catalog-mhskb" Jun 06 22:11:47.959007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.958872 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-envoy\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:11:47.959007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.958887 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-token\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:11:47.959007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.958900 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdv8m\" (UniqueName: \"kubernetes.io/projected/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-kube-api-access-xdv8m\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:11:47.959007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:47.958913 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4-istio-podinfo\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:11:48.059734 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.059657 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpzsc\" (UniqueName: \"kubernetes.io/projected/154a901b-ef4b-48d0-af4c-2458da8b0080-kube-api-access-hpzsc\") pod \"kuadrant-operator-catalog-mhskb\" (UID: \"154a901b-ef4b-48d0-af4c-2458da8b0080\") " pod="kuadrant-system/kuadrant-operator-catalog-mhskb" Jun 06 22:11:48.067403 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.067374 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpzsc\" (UniqueName: \"kubernetes.io/projected/154a901b-ef4b-48d0-af4c-2458da8b0080-kube-api-access-hpzsc\") pod \"kuadrant-operator-catalog-mhskb\" (UID: \"154a901b-ef4b-48d0-af4c-2458da8b0080\") " pod="kuadrant-system/kuadrant-operator-catalog-mhskb" Jun 06 22:11:48.187504 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.187464 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mhskb" Jun 06 22:11:48.319670 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.319646 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mhskb"] Jun 06 22:11:48.322326 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:11:48.322298 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod154a901b_ef4b_48d0_af4c_2458da8b0080.slice/crio-ee46ab14ba413e0045086678b0a08c573ad238ee8989e8fad91785b75c55e07a WatchSource:0}: Error finding container ee46ab14ba413e0045086678b0a08c573ad238ee8989e8fad91785b75c55e07a: Status 404 returned error can't find the container with id ee46ab14ba413e0045086678b0a08c573ad238ee8989e8fad91785b75c55e07a Jun 06 22:11:48.548131 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.548089 2580 generic.go:356] "Generic (PLEG): container finished" podID="05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" containerID="d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5" exitCode=0 Jun 06 22:11:48.548131 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.548133 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" Jun 06 22:11:48.548374 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.548178 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" event={"ID":"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4","Type":"ContainerDied","Data":"d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5"} Jun 06 22:11:48.548374 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.548227 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z" event={"ID":"05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4","Type":"ContainerDied","Data":"5ffd4b495a1f9948796b41a413daa4c43cd85b4944619ad89023d3edb20bcb68"} Jun 06 22:11:48.548374 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.548283 2580 scope.go:117] "RemoveContainer" containerID="d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5" Jun 06 22:11:48.549537 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.549506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mhskb" event={"ID":"154a901b-ef4b-48d0-af4c-2458da8b0080","Type":"ContainerStarted","Data":"ee46ab14ba413e0045086678b0a08c573ad238ee8989e8fad91785b75c55e07a"} Jun 06 22:11:48.557341 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.557317 2580 scope.go:117] "RemoveContainer" containerID="d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5" Jun 06 22:11:48.557609 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:11:48.557588 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5\": container with ID starting with d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5 not found: ID does not exist" containerID="d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5" Jun 06 22:11:48.557684 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.557615 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5"} err="failed to get container status \"d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5\": rpc error: code = NotFound desc = could not find container \"d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5\": container with ID starting with d42e6e7be6562f03072bc8742a108567918fb4a66bc006126254a792b5e452f5 not found: ID does not exist" Jun 06 22:11:48.574386 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.574317 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z"] Jun 06 22:11:48.575745 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.575724 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5df4d9ff6fc4x9z"] Jun 06 22:11:48.632522 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:48.632496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:11:49.490772 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:49.490740 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4" path="/var/lib/kubelet/pods/05f4005c-05a8-4d17-ab3c-b5e6b7e7afe4/volumes" Jun 06 22:11:51.566774 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:51.566738 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mhskb" event={"ID":"154a901b-ef4b-48d0-af4c-2458da8b0080","Type":"ContainerStarted","Data":"d0fe526ffe815500854110c9ebde5ac82bb1adeaf55b5624e44e2c580aba23ea"} Jun 06 22:11:51.588285 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:51.588237 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mhskb" podStartSLOduration=2.362992168 podStartE2EDuration="4.588220822s" totalImagesPullingTime="2.225228654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:11:51.58712507 +0000 UTC m=+538.586104556" watchObservedRunningTime="2026-06-06 22:11:51.588220822 +0000 UTC m=+538.587200306" Jun 06 22:11:58.188101 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:58.188065 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-mhskb" Jun 06 22:11:58.188532 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:58.188125 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-mhskb" Jun 06 22:11:58.210451 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:58.210425 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-mhskb" Jun 06 22:11:58.621633 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:11:58.621557 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-mhskb" Jun 06 22:12:11.401286 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:11.401248 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:11.453247 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:11.453214 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:14.864549 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:14.864512 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:15.466092 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:15.466056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:18.924720 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:18.924677 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9"] Jun 06 22:12:18.929754 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:18.929729 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" Jun 06 22:12:18.936679 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:18.936656 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-pv8xk\"" Jun 06 22:12:18.942408 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:18.942385 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9"] Jun 06 22:12:19.030549 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:19.030515 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66llt\" (UniqueName: \"kubernetes.io/projected/d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e-kube-api-access-66llt\") pod \"authorino-operator-75cc8d5cb7-5j6b9\" (UID: \"d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" Jun 06 22:12:19.131927 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:19.131887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66llt\" (UniqueName: \"kubernetes.io/projected/d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e-kube-api-access-66llt\") pod \"authorino-operator-75cc8d5cb7-5j6b9\" (UID: \"d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" Jun 06 22:12:19.143561 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:19.143537 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66llt\" (UniqueName: \"kubernetes.io/projected/d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e-kube-api-access-66llt\") pod \"authorino-operator-75cc8d5cb7-5j6b9\" (UID: \"d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" Jun 06 22:12:19.244480 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:19.244384 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" Jun 06 22:12:19.385084 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:19.385056 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9"] Jun 06 22:12:19.386633 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:12:19.386608 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31c3b42_d7d6_4fe1_ac60_7e3b00f1198e.slice/crio-9514c52da5edd8cc38df77794ac1fdf7748f9b9873f040ecc41368cd1708771e WatchSource:0}: Error finding container 9514c52da5edd8cc38df77794ac1fdf7748f9b9873f040ecc41368cd1708771e: Status 404 returned error can't find the container with id 9514c52da5edd8cc38df77794ac1fdf7748f9b9873f040ecc41368cd1708771e Jun 06 22:12:19.699743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:19.699708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" event={"ID":"d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e","Type":"ContainerStarted","Data":"9514c52da5edd8cc38df77794ac1fdf7748f9b9873f040ecc41368cd1708771e"} Jun 06 22:12:20.272973 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:20.272936 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:22.196899 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.196866 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s"] Jun 06 22:12:22.201736 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.201716 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:22.204945 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.204924 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6c99q\"" Jun 06 22:12:22.229980 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.229950 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s"] Jun 06 22:12:22.258306 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.258282 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxvd9\" (UniqueName: \"kubernetes.io/projected/a8bc4b9b-c4f4-4a6f-840e-d93246529ac7-kube-api-access-wxvd9\") pod \"limitador-operator-controller-manager-859d7b78bd-tzd5s\" (UID: \"a8bc4b9b-c4f4-4a6f-840e-d93246529ac7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:22.359354 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.359322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxvd9\" (UniqueName: \"kubernetes.io/projected/a8bc4b9b-c4f4-4a6f-840e-d93246529ac7-kube-api-access-wxvd9\") pod \"limitador-operator-controller-manager-859d7b78bd-tzd5s\" (UID: \"a8bc4b9b-c4f4-4a6f-840e-d93246529ac7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:22.369213 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.369189 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxvd9\" (UniqueName: \"kubernetes.io/projected/a8bc4b9b-c4f4-4a6f-840e-d93246529ac7-kube-api-access-wxvd9\") pod \"limitador-operator-controller-manager-859d7b78bd-tzd5s\" (UID: \"a8bc4b9b-c4f4-4a6f-840e-d93246529ac7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:22.403297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.403268 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:22.511710 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.511614 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:22.544372 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.544345 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:22.667057 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.667034 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s"] Jun 06 22:12:22.669116 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:12:22.669088 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8bc4b9b_c4f4_4a6f_840e_d93246529ac7.slice/crio-5c9ab8e0a6ef39a51d27065eee564d0689de183ca6e59186472311599f41d737 WatchSource:0}: Error finding container 5c9ab8e0a6ef39a51d27065eee564d0689de183ca6e59186472311599f41d737: Status 404 returned error can't find the container with id 5c9ab8e0a6ef39a51d27065eee564d0689de183ca6e59186472311599f41d737 Jun 06 22:12:22.717598 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.717571 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" event={"ID":"d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e","Type":"ContainerStarted","Data":"5f8c8cfe6c2e5a51828b4932ff3f8b933595281807f52460a0ecae5dcbf926f4"} Jun 06 22:12:22.717724 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.717630 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" Jun 06 22:12:22.718818 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.718775 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" event={"ID":"a8bc4b9b-c4f4-4a6f-840e-d93246529ac7","Type":"ContainerStarted","Data":"5c9ab8e0a6ef39a51d27065eee564d0689de183ca6e59186472311599f41d737"} Jun 06 22:12:22.744287 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.744246 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" podStartSLOduration=2.470848883 podStartE2EDuration="4.744231454s" totalImagesPullingTime="2.273382571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:12:22.742555161 +0000 UTC m=+569.741534645" watchObservedRunningTime="2026-06-06 22:12:22.744231454 +0000 UTC m=+569.743210937" Jun 06 22:12:22.874524 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:22.874451 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:24.324947 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:24.324901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:24.427620 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:24.427583 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:24.525863 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:24.525834 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:24.731883 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:24.731845 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" event={"ID":"a8bc4b9b-c4f4-4a6f-840e-d93246529ac7","Type":"ContainerStarted","Data":"ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824"} Jun 06 22:12:24.732189 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:24.732133 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:24.757736 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:24.757676 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:24.768914 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:24.768867 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" podStartSLOduration=0.921369057 podStartE2EDuration="2.76885059s" totalImagesPullingTime="1.847481533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:12:24.767656324 +0000 UTC m=+571.766635809" watchObservedRunningTime="2026-06-06 22:12:24.76885059 +0000 UTC m=+571.767830074" Jun 06 22:12:25.077407 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:25.077363 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:25.176711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:25.176682 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:26.510474 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:26.510434 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:27.550340 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:27.550308 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:27.648337 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:27.648307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:28.080894 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:28.080840 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:29.318709 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:29.318672 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:31.276287 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:31.276245 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:31.376891 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:31.376844 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:31.667549 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:31.667502 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:31.732882 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:31.732851 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:33.726448 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:33.726421 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5j6b9" Jun 06 22:12:35.172485 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:35.172450 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:35.259079 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:35.259007 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:35.361443 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:35.361408 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:35.464351 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:35.464252 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:35.738718 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:35.738642 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:38.470016 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:38.469968 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:38.520154 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:38.520118 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:44.923599 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:44.923551 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:45.023558 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.023526 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:45.056478 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.056444 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2"] Jun 06 22:12:45.059845 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.059823 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:12:45.062449 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.062432 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l7rcj\"" Jun 06 22:12:45.073722 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.073697 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2"] Jun 06 22:12:45.096562 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.096533 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s"] Jun 06 22:12:45.096782 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.096758 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" podUID="a8bc4b9b-c4f4-4a6f-840e-d93246529ac7" containerName="manager" containerID="cri-o://ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824" gracePeriod=2 Jun 06 22:12:45.113955 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.113930 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s"] Jun 06 22:12:45.117084 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:12:45.117052 2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"a8bc4b9b-c4f4-4a6f-840e-d93246529ac7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-tzd5s\": pods \"limitador-operator-controller-manager-859d7b78bd-tzd5s\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:45.156917 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.156884 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4dn9\" (UniqueName: \"kubernetes.io/projected/bb5ab631-fa4c-40c2-8469-841109b5adda-kube-api-access-r4dn9\") pod \"kuadrant-operator-controller-manager-55467d9f7-zvfj2\" (UID: \"bb5ab631-fa4c-40c2-8469-841109b5adda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:12:45.157051 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.156963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb5ab631-fa4c-40c2-8469-841109b5adda-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zvfj2\" (UID: \"bb5ab631-fa4c-40c2-8469-841109b5adda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:12:45.258052 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.258015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4dn9\" (UniqueName: \"kubernetes.io/projected/bb5ab631-fa4c-40c2-8469-841109b5adda-kube-api-access-r4dn9\") pod \"kuadrant-operator-controller-manager-55467d9f7-zvfj2\" (UID: \"bb5ab631-fa4c-40c2-8469-841109b5adda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:12:45.258197 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.258075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb5ab631-fa4c-40c2-8469-841109b5adda-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zvfj2\" (UID: \"bb5ab631-fa4c-40c2-8469-841109b5adda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:12:45.258472 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.258452 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb5ab631-fa4c-40c2-8469-841109b5adda-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zvfj2\" (UID: \"bb5ab631-fa4c-40c2-8469-841109b5adda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:12:45.267978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.267947 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4dn9\" (UniqueName: \"kubernetes.io/projected/bb5ab631-fa4c-40c2-8469-841109b5adda-kube-api-access-r4dn9\") pod \"kuadrant-operator-controller-manager-55467d9f7-zvfj2\" (UID: \"bb5ab631-fa4c-40c2-8469-841109b5adda\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:12:45.344747 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.344719 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:45.347246 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:12:45.347217 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tzd5s\" is forbidden: User \"system:node:ip-10-0-139-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-115.ec2.internal' and this object" podUID="a8bc4b9b-c4f4-4a6f-840e-d93246529ac7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:45.370599 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.370571 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:12:45.459176 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.459147 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxvd9\" (UniqueName: \"kubernetes.io/projected/a8bc4b9b-c4f4-4a6f-840e-d93246529ac7-kube-api-access-wxvd9\") pod \"a8bc4b9b-c4f4-4a6f-840e-d93246529ac7\" (UID: \"a8bc4b9b-c4f4-4a6f-840e-d93246529ac7\") " Jun 06 22:12:45.461671 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.461636 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8bc4b9b-c4f4-4a6f-840e-d93246529ac7-kube-api-access-wxvd9" (OuterVolumeSpecName: "kube-api-access-wxvd9") pod "a8bc4b9b-c4f4-4a6f-840e-d93246529ac7" (UID: "a8bc4b9b-c4f4-4a6f-840e-d93246529ac7"). InnerVolumeSpecName "kube-api-access-wxvd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:12:45.494280 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.494204 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8bc4b9b-c4f4-4a6f-840e-d93246529ac7" path="/var/lib/kubelet/pods/a8bc4b9b-c4f4-4a6f-840e-d93246529ac7/volumes" Jun 06 22:12:45.503652 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.503630 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2"] Jun 06 22:12:45.506477 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:12:45.506448 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5ab631_fa4c_40c2_8469_841109b5adda.slice/crio-b2fef246bae16aeda43a73db725c611de057a48a172c8700a6c51c6f55bbed78 WatchSource:0}: Error finding container b2fef246bae16aeda43a73db725c611de057a48a172c8700a6c51c6f55bbed78: Status 404 returned error can't find the container with id b2fef246bae16aeda43a73db725c611de057a48a172c8700a6c51c6f55bbed78 Jun 06 22:12:45.561419 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.561390 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxvd9\" (UniqueName: \"kubernetes.io/projected/a8bc4b9b-c4f4-4a6f-840e-d93246529ac7-kube-api-access-wxvd9\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:12:45.830453 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.830359 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" event={"ID":"bb5ab631-fa4c-40c2-8469-841109b5adda","Type":"ContainerStarted","Data":"b2fef246bae16aeda43a73db725c611de057a48a172c8700a6c51c6f55bbed78"} Jun 06 22:12:45.831583 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.831556 2580 generic.go:356] "Generic (PLEG): container finished" podID="a8bc4b9b-c4f4-4a6f-840e-d93246529ac7" containerID="ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824" exitCode=0 Jun 06 22:12:45.831704 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.831612 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tzd5s" Jun 06 22:12:45.831704 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.831647 2580 scope.go:117] "RemoveContainer" containerID="ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824" Jun 06 22:12:45.839969 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.839945 2580 scope.go:117] "RemoveContainer" containerID="ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824" Jun 06 22:12:45.840217 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:12:45.840201 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824\": container with ID starting with ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824 not found: ID does not exist" containerID="ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824" Jun 06 22:12:45.840276 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:45.840236 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824"} err="failed to get container status \"ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824\": rpc error: code = NotFound desc = could not find container \"ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824\": container with ID starting with ae630299c1096f7c482a600e93807aa6d740018455455295b94b557e0ae7a824 not found: ID does not exist" Jun 06 22:12:49.489054 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:49.489026 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:49.855312 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:49.855219 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" event={"ID":"bb5ab631-fa4c-40c2-8469-841109b5adda","Type":"ContainerStarted","Data":"ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2"} Jun 06 22:12:49.855478 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:49.855360 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:12:49.877722 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:49.877676 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" podStartSLOduration=1.514931306 podStartE2EDuration="4.877662141s" totalImagesPullingTime="3.362730835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:12:49.875324358 +0000 UTC m=+596.874303842" watchObservedRunningTime="2026-06-06 22:12:49.877662141 +0000 UTC m=+596.876641680" Jun 06 22:12:50.491411 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:50.491367 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:50.590001 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:50.589959 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:51.038800 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:51.038767 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:51.107442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:51.107407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:53.464628 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:53.464600 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:12:53.465473 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:53.465449 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:12:53.471988 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:53.471911 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:12:53.472700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:53.472680 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:12:54.444819 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:54.444782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:54.546445 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:54.546415 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:56.640832 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:56.640775 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:58.054188 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:58.054153 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:12:58.155180 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:12:58.155145 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:00.862200 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:00.862164 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:13:01.128339 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.128263 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9"] Jun 06 22:13:01.128620 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.128608 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8bc4b9b-c4f4-4a6f-840e-d93246529ac7" containerName="manager" Jun 06 22:13:01.128663 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.128621 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8bc4b9b-c4f4-4a6f-840e-d93246529ac7" containerName="manager" Jun 06 22:13:01.128701 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.128692 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8bc4b9b-c4f4-4a6f-840e-d93246529ac7" containerName="manager" Jun 06 22:13:01.133145 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.133123 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:01.143766 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.143744 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9"] Jun 06 22:13:01.306654 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.306619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7734961d-c56c-4b7f-a456-42a70e164474-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mk7j9\" (UID: \"7734961d-c56c-4b7f-a456-42a70e164474\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:01.306847 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.306666 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lw2d\" (UniqueName: \"kubernetes.io/projected/7734961d-c56c-4b7f-a456-42a70e164474-kube-api-access-9lw2d\") pod \"kuadrant-operator-controller-manager-86878c769f-mk7j9\" (UID: \"7734961d-c56c-4b7f-a456-42a70e164474\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:01.407196 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.407163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lw2d\" (UniqueName: \"kubernetes.io/projected/7734961d-c56c-4b7f-a456-42a70e164474-kube-api-access-9lw2d\") pod \"kuadrant-operator-controller-manager-86878c769f-mk7j9\" (UID: \"7734961d-c56c-4b7f-a456-42a70e164474\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:01.407341 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.407248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7734961d-c56c-4b7f-a456-42a70e164474-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mk7j9\" (UID: \"7734961d-c56c-4b7f-a456-42a70e164474\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:01.407580 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.407563 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7734961d-c56c-4b7f-a456-42a70e164474-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mk7j9\" (UID: \"7734961d-c56c-4b7f-a456-42a70e164474\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:01.415513 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.415484 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lw2d\" (UniqueName: \"kubernetes.io/projected/7734961d-c56c-4b7f-a456-42a70e164474-kube-api-access-9lw2d\") pod \"kuadrant-operator-controller-manager-86878c769f-mk7j9\" (UID: \"7734961d-c56c-4b7f-a456-42a70e164474\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:01.444327 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.444297 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:01.591413 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.591390 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9"] Jun 06 22:13:01.594050 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:13:01.594010 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7734961d_c56c_4b7f_a456_42a70e164474.slice/crio-7ba00663440f8df13b819a8ffbfe9d8ba4800399022371dadff68a02c0619962 WatchSource:0}: Error finding container 7ba00663440f8df13b819a8ffbfe9d8ba4800399022371dadff68a02c0619962: Status 404 returned error can't find the container with id 7ba00663440f8df13b819a8ffbfe9d8ba4800399022371dadff68a02c0619962 Jun 06 22:13:01.756885 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.756854 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:01.849363 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.849331 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:01.913777 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.913748 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" event={"ID":"7734961d-c56c-4b7f-a456-42a70e164474","Type":"ContainerStarted","Data":"3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36"} Jun 06 22:13:01.914174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.913816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" event={"ID":"7734961d-c56c-4b7f-a456-42a70e164474","Type":"ContainerStarted","Data":"7ba00663440f8df13b819a8ffbfe9d8ba4800399022371dadff68a02c0619962"} Jun 06 22:13:01.914174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.913860 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:01.938170 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:01.938085 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" podStartSLOduration=0.93806896 podStartE2EDuration="938.06896ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:13:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:13:01.936941594 +0000 UTC m=+608.935921079" watchObservedRunningTime="2026-06-06 22:13:01.93806896 +0000 UTC m=+608.937048443" Jun 06 22:13:03.726313 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:03.726288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:03.820905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:03.820872 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:12.921362 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:12.921284 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:13:12.981332 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:12.981295 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2"] Jun 06 22:13:12.981896 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:12.981839 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" podUID="bb5ab631-fa4c-40c2-8469-841109b5adda" containerName="manager" containerID="cri-o://ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2" gracePeriod=10 Jun 06 22:13:13.240510 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.240484 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:13:13.308759 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.308729 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4dn9\" (UniqueName: \"kubernetes.io/projected/bb5ab631-fa4c-40c2-8469-841109b5adda-kube-api-access-r4dn9\") pod \"bb5ab631-fa4c-40c2-8469-841109b5adda\" (UID: \"bb5ab631-fa4c-40c2-8469-841109b5adda\") " Jun 06 22:13:13.308932 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.308838 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb5ab631-fa4c-40c2-8469-841109b5adda-extensions-socket-volume\") pod \"bb5ab631-fa4c-40c2-8469-841109b5adda\" (UID: \"bb5ab631-fa4c-40c2-8469-841109b5adda\") " Jun 06 22:13:13.309219 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.309197 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb5ab631-fa4c-40c2-8469-841109b5adda-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bb5ab631-fa4c-40c2-8469-841109b5adda" (UID: "bb5ab631-fa4c-40c2-8469-841109b5adda"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:13:13.310863 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.310843 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5ab631-fa4c-40c2-8469-841109b5adda-kube-api-access-r4dn9" (OuterVolumeSpecName: "kube-api-access-r4dn9") pod "bb5ab631-fa4c-40c2-8469-841109b5adda" (UID: "bb5ab631-fa4c-40c2-8469-841109b5adda"). InnerVolumeSpecName "kube-api-access-r4dn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:13:13.409474 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.409441 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4dn9\" (UniqueName: \"kubernetes.io/projected/bb5ab631-fa4c-40c2-8469-841109b5adda-kube-api-access-r4dn9\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:13:13.409474 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.409467 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb5ab631-fa4c-40c2-8469-841109b5adda-extensions-socket-volume\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:13:13.974582 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.974545 2580 generic.go:356] "Generic (PLEG): container finished" podID="bb5ab631-fa4c-40c2-8469-841109b5adda" containerID="ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2" exitCode=0 Jun 06 22:13:13.974998 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.974594 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" Jun 06 22:13:13.974998 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.974637 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" event={"ID":"bb5ab631-fa4c-40c2-8469-841109b5adda","Type":"ContainerDied","Data":"ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2"} Jun 06 22:13:13.974998 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.974694 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2" event={"ID":"bb5ab631-fa4c-40c2-8469-841109b5adda","Type":"ContainerDied","Data":"b2fef246bae16aeda43a73db725c611de057a48a172c8700a6c51c6f55bbed78"} Jun 06 22:13:13.974998 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.974723 2580 scope.go:117] "RemoveContainer" containerID="ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2" Jun 06 22:13:13.985746 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.985560 2580 scope.go:117] "RemoveContainer" containerID="ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2" Jun 06 22:13:13.985892 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:13:13.985872 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2\": container with ID starting with ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2 not found: ID does not exist" containerID="ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2" Jun 06 22:13:13.985938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.985901 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2"} err="failed to get container status \"ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2\": rpc error: code = NotFound desc = could not find container \"ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2\": container with ID starting with ae0833d8e59f3e3869a2a4a50a7aae5d63a83d356c1de693fc1a256f995983b2 not found: ID does not exist" Jun 06 22:13:13.997906 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:13.997877 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2"] Jun 06 22:13:14.006354 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:14.006329 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zvfj2"] Jun 06 22:13:15.488543 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:15.488510 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb5ab631-fa4c-40c2-8469-841109b5adda" path="/var/lib/kubelet/pods/bb5ab631-fa4c-40c2-8469-841109b5adda/volumes" Jun 06 22:13:17.143536 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.143503 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w"] Jun 06 22:13:17.143913 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.143872 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5ab631-fa4c-40c2-8469-841109b5adda" containerName="manager" Jun 06 22:13:17.143913 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.143883 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5ab631-fa4c-40c2-8469-841109b5adda" containerName="manager" Jun 06 22:13:17.144188 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.143959 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb5ab631-fa4c-40c2-8469-841109b5adda" containerName="manager" Jun 06 22:13:17.146859 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.146833 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.149570 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.149551 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bmgnb\"" Jun 06 22:13:17.164064 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.164032 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w"] Jun 06 22:13:17.194256 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.194224 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:17.238480 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.238449 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.238480 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.238479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gm78\" (UniqueName: \"kubernetes.io/projected/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-kube-api-access-9gm78\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.238680 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.238501 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.238680 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.238522 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.238680 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.238619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.238784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.238681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.238784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.238708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.238784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.238731 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.238784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.238765 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.304570 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.304535 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:17.339695 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.339660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.339921 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.339702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.339921 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.339857 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.339921 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.339911 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340095 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.339978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340095 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.340004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gm78\" (UniqueName: \"kubernetes.io/projected/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-kube-api-access-9gm78\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340095 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.340052 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340253 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.340233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340306 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.340256 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340369 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.340348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340501 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.340463 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340672 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.340651 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340740 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.340668 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.340740 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.340687 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.342211 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.342190 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.342463 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.342445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.349882 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.349789 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.350017 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.349944 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gm78\" (UniqueName: \"kubernetes.io/projected/aeb14052-2c78-4059-9b3d-4d0c9bec4f4e-kube-api-access-9gm78\") pod \"maas-default-gateway-openshift-default-687ff6996-zrt7w\" (UID: \"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.460833 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.460731 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:17.619439 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.615640 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w"] Jun 06 22:13:17.622909 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:13:17.622877 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb14052_2c78_4059_9b3d_4d0c9bec4f4e.slice/crio-e9b3e80a376ee67458766cd65ddabc1ebd9e8db7644df88c71f03c991841e719 WatchSource:0}: Error finding container e9b3e80a376ee67458766cd65ddabc1ebd9e8db7644df88c71f03c991841e719: Status 404 returned error can't find the container with id e9b3e80a376ee67458766cd65ddabc1ebd9e8db7644df88c71f03c991841e719 Jun 06 22:13:17.624769 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.624739 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:13:17.624854 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.624816 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:13:17.624854 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.624849 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:13:17.997385 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.997349 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" event={"ID":"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e","Type":"ContainerStarted","Data":"4f027a554683f891ba35569813d96beec8abf9ca34fdef88786abf30ea89ec64"} Jun 06 22:13:17.997561 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:17.997400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" event={"ID":"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e","Type":"ContainerStarted","Data":"e9b3e80a376ee67458766cd65ddabc1ebd9e8db7644df88c71f03c991841e719"} Jun 06 22:13:18.024734 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:18.024677 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" podStartSLOduration=1.024654509 podStartE2EDuration="1.024654509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:13:18.02217719 +0000 UTC m=+625.021156677" watchObservedRunningTime="2026-06-06 22:13:18.024654509 +0000 UTC m=+625.023633996" Jun 06 22:13:18.461427 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:18.461386 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:18.466204 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:18.466177 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:19.002960 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:19.002927 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:19.003819 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:19.003788 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:13:20.684572 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:20.684538 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:31.141170 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:31.141137 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:31.250776 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:31.250718 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:31.367409 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:31.367373 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:31.999315 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:31.999279 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-dqtvg"] Jun 06 22:13:32.003046 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.003023 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dqtvg" Jun 06 22:13:32.005773 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.005754 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rn25q\"" Jun 06 22:13:32.014749 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.014725 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-dqtvg"] Jun 06 22:13:32.040519 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.040469 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:32.071166 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.071131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rk48\" (UniqueName: \"kubernetes.io/projected/d142baa0-0fbd-4f88-8bcf-c201e13b2117-kube-api-access-4rk48\") pod \"authorino-58cd955675-dqtvg\" (UID: \"d142baa0-0fbd-4f88-8bcf-c201e13b2117\") " pod="kuadrant-system/authorino-58cd955675-dqtvg" Jun 06 22:13:32.115637 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.115604 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:32.171773 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.171737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rk48\" (UniqueName: \"kubernetes.io/projected/d142baa0-0fbd-4f88-8bcf-c201e13b2117-kube-api-access-4rk48\") pod \"authorino-58cd955675-dqtvg\" (UID: \"d142baa0-0fbd-4f88-8bcf-c201e13b2117\") " pod="kuadrant-system/authorino-58cd955675-dqtvg" Jun 06 22:13:32.180181 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.180155 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rk48\" (UniqueName: \"kubernetes.io/projected/d142baa0-0fbd-4f88-8bcf-c201e13b2117-kube-api-access-4rk48\") pod \"authorino-58cd955675-dqtvg\" (UID: \"d142baa0-0fbd-4f88-8bcf-c201e13b2117\") " pod="kuadrant-system/authorino-58cd955675-dqtvg" Jun 06 22:13:32.313742 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.313647 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dqtvg" Jun 06 22:13:32.468653 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:32.468628 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-dqtvg"] Jun 06 22:13:32.470610 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:13:32.470581 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd142baa0_0fbd_4f88_8bcf_c201e13b2117.slice/crio-1649e1dff453aedc382f83180ce99ab520b785ff3c09cd3c612a156f93a4f156 WatchSource:0}: Error finding container 1649e1dff453aedc382f83180ce99ab520b785ff3c09cd3c612a156f93a4f156: Status 404 returned error can't find the container with id 1649e1dff453aedc382f83180ce99ab520b785ff3c09cd3c612a156f93a4f156 Jun 06 22:13:33.069201 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:33.069160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dqtvg" event={"ID":"d142baa0-0fbd-4f88-8bcf-c201e13b2117","Type":"ContainerStarted","Data":"1649e1dff453aedc382f83180ce99ab520b785ff3c09cd3c612a156f93a4f156"} Jun 06 22:13:36.085611 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:36.085572 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dqtvg" event={"ID":"d142baa0-0fbd-4f88-8bcf-c201e13b2117","Type":"ContainerStarted","Data":"f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a"} Jun 06 22:13:36.101292 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:36.101201 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-dqtvg" podStartSLOduration=1.726652968 podStartE2EDuration="5.101186136s" totalImagesPullingTime="3.374533168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:13:36.100889411 +0000 UTC m=+643.099868896" watchObservedRunningTime="2026-06-06 22:13:36.101186136 +0000 UTC m=+643.100165632" Jun 06 22:13:37.501580 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:37.501548 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-dqtvg"] Jun 06 22:13:38.095182 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:38.095142 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-dqtvg" podUID="d142baa0-0fbd-4f88-8bcf-c201e13b2117" containerName="authorino" containerID="cri-o://f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a" gracePeriod=30 Jun 06 22:13:38.347155 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:38.347093 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dqtvg" Jun 06 22:13:38.435264 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:38.435234 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rk48\" (UniqueName: \"kubernetes.io/projected/d142baa0-0fbd-4f88-8bcf-c201e13b2117-kube-api-access-4rk48\") pod \"d142baa0-0fbd-4f88-8bcf-c201e13b2117\" (UID: \"d142baa0-0fbd-4f88-8bcf-c201e13b2117\") " Jun 06 22:13:38.437337 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:38.437304 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d142baa0-0fbd-4f88-8bcf-c201e13b2117-kube-api-access-4rk48" (OuterVolumeSpecName: "kube-api-access-4rk48") pod "d142baa0-0fbd-4f88-8bcf-c201e13b2117" (UID: "d142baa0-0fbd-4f88-8bcf-c201e13b2117"). InnerVolumeSpecName "kube-api-access-4rk48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:13:38.536630 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:38.536599 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rk48\" (UniqueName: \"kubernetes.io/projected/d142baa0-0fbd-4f88-8bcf-c201e13b2117-kube-api-access-4rk48\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:13:39.101399 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.101365 2580 generic.go:356] "Generic (PLEG): container finished" podID="d142baa0-0fbd-4f88-8bcf-c201e13b2117" containerID="f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a" exitCode=0 Jun 06 22:13:39.101569 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.101407 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dqtvg" Jun 06 22:13:39.101569 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.101417 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dqtvg" event={"ID":"d142baa0-0fbd-4f88-8bcf-c201e13b2117","Type":"ContainerDied","Data":"f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a"} Jun 06 22:13:39.101569 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.101449 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dqtvg" event={"ID":"d142baa0-0fbd-4f88-8bcf-c201e13b2117","Type":"ContainerDied","Data":"1649e1dff453aedc382f83180ce99ab520b785ff3c09cd3c612a156f93a4f156"} Jun 06 22:13:39.101569 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.101466 2580 scope.go:117] "RemoveContainer" containerID="f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a" Jun 06 22:13:39.110176 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.110157 2580 scope.go:117] "RemoveContainer" containerID="f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a" Jun 06 22:13:39.110458 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:13:39.110439 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a\": container with ID starting with f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a not found: ID does not exist" containerID="f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a" Jun 06 22:13:39.110510 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.110470 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a"} err="failed to get container status \"f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a\": rpc error: code = NotFound desc = could not find container \"f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a\": container with ID starting with f7a67f695a1013ad4b4e56183224864cdd908a4573f6b7d49a820df0cd9e375a not found: ID does not exist" Jun 06 22:13:39.122544 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.122522 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-dqtvg"] Jun 06 22:13:39.128182 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.128160 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-dqtvg"] Jun 06 22:13:39.488254 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:39.488223 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d142baa0-0fbd-4f88-8bcf-c201e13b2117" path="/var/lib/kubelet/pods/d142baa0-0fbd-4f88-8bcf-c201e13b2117/volumes" Jun 06 22:13:52.467467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:52.467426 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:13:59.038525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:13:59.038449 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:14:02.097115 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:02.097078 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:14:02.240296 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:02.240256 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:14:02.364745 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:02.364646 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:14:25.765250 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:25.765214 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:14:25.866521 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:25.866490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:14:26.059331 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.059240 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 06 22:14:26.059704 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.059685 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d142baa0-0fbd-4f88-8bcf-c201e13b2117" containerName="authorino" Jun 06 22:14:26.059785 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.059707 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d142baa0-0fbd-4f88-8bcf-c201e13b2117" containerName="authorino" Jun 06 22:14:26.059887 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.059841 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d142baa0-0fbd-4f88-8bcf-c201e13b2117" containerName="authorino" Jun 06 22:14:26.062880 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.062860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 06 22:14:26.065450 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.065429 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 06 22:14:26.065575 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.065435 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 06 22:14:26.066543 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.066525 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-4d87f\"" Jun 06 22:14:26.066652 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.066563 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 06 22:14:26.071666 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.071648 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 06 22:14:26.092932 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.092900 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp9dx\" (UniqueName: \"kubernetes.io/projected/5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36-kube-api-access-gp9dx\") pod \"maas-keycloak-0\" (UID: \"5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36\") " pod="keycloak-system/maas-keycloak-0" Jun 06 22:14:26.193706 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.193678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp9dx\" (UniqueName: \"kubernetes.io/projected/5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36-kube-api-access-gp9dx\") pod \"maas-keycloak-0\" (UID: \"5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36\") " pod="keycloak-system/maas-keycloak-0" Jun 06 22:14:26.201988 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.201964 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp9dx\" (UniqueName: \"kubernetes.io/projected/5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36-kube-api-access-gp9dx\") pod \"maas-keycloak-0\" (UID: \"5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36\") " pod="keycloak-system/maas-keycloak-0" Jun 06 22:14:26.373425 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.373335 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 06 22:14:26.508469 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:26.508442 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 06 22:14:26.509962 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:14:26.509938 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2491ce_b6ae_4c63_bfcd_333dcf7f1c36.slice/crio-35f9a245c62e85031661c8527683d15c6c7a43856ac67b25f933014e69812853 WatchSource:0}: Error finding container 35f9a245c62e85031661c8527683d15c6c7a43856ac67b25f933014e69812853: Status 404 returned error can't find the container with id 35f9a245c62e85031661c8527683d15c6c7a43856ac67b25f933014e69812853 Jun 06 22:14:27.343662 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:27.343587 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36","Type":"ContainerStarted","Data":"35f9a245c62e85031661c8527683d15c6c7a43856ac67b25f933014e69812853"} Jun 06 22:14:31.369956 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:31.369852 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36","Type":"ContainerStarted","Data":"9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392"} Jun 06 22:14:31.373945 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:31.373919 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 06 22:14:31.375218 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:31.375189 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:31.390289 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:31.390231 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.929822978 podStartE2EDuration="5.390212807s" totalImagesPullingTime="4.460389829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:14:31.387683851 +0000 UTC m=+698.386663362" watchObservedRunningTime="2026-06-06 22:14:31.390212807 +0000 UTC m=+698.389192295" Jun 06 22:14:32.374217 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:32.374173 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:33.374344 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:33.374289 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:34.374482 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:34.374428 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:35.373780 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:35.373734 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:36.374851 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:36.374786 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:36.375365 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:36.375282 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 06 22:14:37.374105 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:37.374065 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:38.374512 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:38.374459 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:38.460474 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:38.460432 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:14:38.516645 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:38.516601 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:14:39.374605 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:39.374549 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:40.374054 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:40.373999 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:41.374708 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:41.374654 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:42.374549 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:42.374506 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:43.374228 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:43.374129 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.37:9000/health/started\": dial tcp 10.133.0.37:9000: connect: connection refused" Jun 06 22:14:44.499745 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:44.499696 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 06 22:14:44.517394 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:44.517342 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:14:54.506221 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:54.506189 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 06 22:14:55.141691 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:55.141645 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:14:55.531694 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:14:55.531592 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:00.335473 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.335442 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:00.339935 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.339902 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c984c879d-v6hc9"] Jun 06 22:15:00.345929 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.345908 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:00.348586 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.348560 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 06 22:15:00.348700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.348566 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 06 22:15:00.348700 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.348590 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wgdtv\"" Jun 06 22:15:00.352771 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.352743 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c984c879d-v6hc9"] Jun 06 22:15:00.433949 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.433914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:00.447297 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.447267 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm4nw\" (UniqueName: \"kubernetes.io/projected/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-kube-api-access-cm4nw\") pod \"maas-controller-c984c879d-v6hc9\" (UID: \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\") " pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:00.447447 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.447359 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-webhook-cert\") pod \"maas-controller-c984c879d-v6hc9\" (UID: \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\") " pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:00.490907 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.490878 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-844f958585-6dchl"] Jun 06 22:15:00.492773 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.492749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:00.493655 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.493639 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:00.503982 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.503958 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-844f958585-6dchl"] Jun 06 22:15:00.548493 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.548463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-webhook-cert\") pod \"maas-controller-c984c879d-v6hc9\" (UID: \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\") " pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:00.548727 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.548513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-webhook-cert\") pod \"maas-controller-844f958585-6dchl\" (UID: \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\") " pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:00.548727 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.548568 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4nw\" (UniqueName: \"kubernetes.io/projected/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-kube-api-access-cm4nw\") pod \"maas-controller-c984c879d-v6hc9\" (UID: \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\") " pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:00.548727 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.548662 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8784n\" (UniqueName: \"kubernetes.io/projected/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-kube-api-access-8784n\") pod \"maas-controller-844f958585-6dchl\" (UID: \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\") " pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:00.550986 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.550963 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-webhook-cert\") pod \"maas-controller-c984c879d-v6hc9\" (UID: \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\") " pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:00.557241 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.557221 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm4nw\" (UniqueName: \"kubernetes.io/projected/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-kube-api-access-cm4nw\") pod \"maas-controller-c984c879d-v6hc9\" (UID: \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\") " pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:00.588671 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.588580 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:00.649733 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.649699 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8784n\" (UniqueName: \"kubernetes.io/projected/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-kube-api-access-8784n\") pod \"maas-controller-844f958585-6dchl\" (UID: \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\") " pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:00.650124 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.649759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-webhook-cert\") pod \"maas-controller-844f958585-6dchl\" (UID: \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\") " pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:00.652040 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.652010 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-webhook-cert\") pod \"maas-controller-844f958585-6dchl\" (UID: \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\") " pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:00.657296 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.657274 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:00.658474 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.658455 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8784n\" (UniqueName: \"kubernetes.io/projected/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-kube-api-access-8784n\") pod \"maas-controller-844f958585-6dchl\" (UID: \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\") " pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:00.789308 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.789281 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c984c879d-v6hc9"] Jun 06 22:15:00.791894 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:15:00.791867 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod742f19a7_ef72_4ba3_bc56_bc4e757bbe72.slice/crio-a6b4d72c8f741339cd77100a8d962122ee0317bd07a13c3db655a7d7d5a55ffe WatchSource:0}: Error finding container a6b4d72c8f741339cd77100a8d962122ee0317bd07a13c3db655a7d7d5a55ffe: Status 404 returned error can't find the container with id a6b4d72c8f741339cd77100a8d962122ee0317bd07a13c3db655a7d7d5a55ffe Jun 06 22:15:00.804316 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.804293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:00.941018 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:00.940985 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-844f958585-6dchl"] Jun 06 22:15:00.943963 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:15:00.943933 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod496c6e85_8cb6_4100_9f0b_408d4f3a75eb.slice/crio-a4b268ae635124fa7e086b0d46095aa5e6b157b21b33f5f43c1c1bec4dd76ede WatchSource:0}: Error finding container a4b268ae635124fa7e086b0d46095aa5e6b157b21b33f5f43c1c1bec4dd76ede: Status 404 returned error can't find the container with id a4b268ae635124fa7e086b0d46095aa5e6b157b21b33f5f43c1c1bec4dd76ede Jun 06 22:15:01.560899 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:01.560857 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844f958585-6dchl" event={"ID":"496c6e85-8cb6-4100-9f0b-408d4f3a75eb","Type":"ContainerStarted","Data":"a4b268ae635124fa7e086b0d46095aa5e6b157b21b33f5f43c1c1bec4dd76ede"} Jun 06 22:15:01.564467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:01.564436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c984c879d-v6hc9" event={"ID":"742f19a7-ef72-4ba3-bc56-bc4e757bbe72","Type":"ContainerStarted","Data":"a6b4d72c8f741339cd77100a8d962122ee0317bd07a13c3db655a7d7d5a55ffe"} Jun 06 22:15:03.791263 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:03.791168 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:03.879412 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:03.879373 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:04.588079 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:04.588042 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844f958585-6dchl" event={"ID":"496c6e85-8cb6-4100-9f0b-408d4f3a75eb","Type":"ContainerStarted","Data":"3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4"} Jun 06 22:15:04.588393 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:04.588137 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:04.589506 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:04.589479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c984c879d-v6hc9" event={"ID":"742f19a7-ef72-4ba3-bc56-bc4e757bbe72","Type":"ContainerStarted","Data":"e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946"} Jun 06 22:15:04.589629 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:04.589534 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:04.612301 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:04.612247 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-844f958585-6dchl" podStartSLOduration=2.030422109 podStartE2EDuration="4.612230633s" totalImagesPullingTime="2.581808524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:15:04.609907317 +0000 UTC m=+731.608886802" watchObservedRunningTime="2026-06-06 22:15:04.612230633 +0000 UTC m=+731.611210094" Jun 06 22:15:04.639169 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:04.639114 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c984c879d-v6hc9" podStartSLOduration=1.8968635919999999 podStartE2EDuration="4.639095247s" totalImagesPullingTime="2.742231655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:15:04.638080588 +0000 UTC m=+731.637060076" watchObservedRunningTime="2026-06-06 22:15:04.639095247 +0000 UTC m=+731.638074730" Jun 06 22:15:15.599567 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:15.599536 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:15.600024 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:15.599971 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:15:15.658506 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:15.658472 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c984c879d-v6hc9"] Jun 06 22:15:15.658759 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:15.658725 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-c984c879d-v6hc9" podUID="742f19a7-ef72-4ba3-bc56-bc4e757bbe72" containerName="manager" containerID="cri-o://e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946" gracePeriod=30 Jun 06 22:15:15.924325 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:15.924296 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:15.996757 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:15.996721 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm4nw\" (UniqueName: \"kubernetes.io/projected/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-kube-api-access-cm4nw\") pod \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\" (UID: \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\") " Jun 06 22:15:15.996933 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:15.996877 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-webhook-cert\") pod \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\" (UID: \"742f19a7-ef72-4ba3-bc56-bc4e757bbe72\") " Jun 06 22:15:15.999006 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:15.998971 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-kube-api-access-cm4nw" (OuterVolumeSpecName: "kube-api-access-cm4nw") pod "742f19a7-ef72-4ba3-bc56-bc4e757bbe72" (UID: "742f19a7-ef72-4ba3-bc56-bc4e757bbe72"). InnerVolumeSpecName "kube-api-access-cm4nw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:15:15.999006 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:15.998993 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "742f19a7-ef72-4ba3-bc56-bc4e757bbe72" (UID: "742f19a7-ef72-4ba3-bc56-bc4e757bbe72"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:15:16.098244 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.098196 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-webhook-cert\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:15:16.098244 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.098238 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm4nw\" (UniqueName: \"kubernetes.io/projected/742f19a7-ef72-4ba3-bc56-bc4e757bbe72-kube-api-access-cm4nw\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:15:16.649361 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.649323 2580 generic.go:356] "Generic (PLEG): container finished" podID="742f19a7-ef72-4ba3-bc56-bc4e757bbe72" containerID="e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946" exitCode=0 Jun 06 22:15:16.649747 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.649368 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c984c879d-v6hc9" Jun 06 22:15:16.649747 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.649405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c984c879d-v6hc9" event={"ID":"742f19a7-ef72-4ba3-bc56-bc4e757bbe72","Type":"ContainerDied","Data":"e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946"} Jun 06 22:15:16.649747 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.649456 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c984c879d-v6hc9" event={"ID":"742f19a7-ef72-4ba3-bc56-bc4e757bbe72","Type":"ContainerDied","Data":"a6b4d72c8f741339cd77100a8d962122ee0317bd07a13c3db655a7d7d5a55ffe"} Jun 06 22:15:16.649747 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.649477 2580 scope.go:117] "RemoveContainer" containerID="e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946" Jun 06 22:15:16.658524 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.658509 2580 scope.go:117] "RemoveContainer" containerID="e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946" Jun 06 22:15:16.658753 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:15:16.658738 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946\": container with ID starting with e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946 not found: ID does not exist" containerID="e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946" Jun 06 22:15:16.658798 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.658760 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946"} err="failed to get container status \"e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946\": rpc error: code = NotFound desc = could not find container \"e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946\": container with ID starting with e1a23c8e44d65998dd16d7de03141726ab04e14385a793c5562ce71e6ac40946 not found: ID does not exist" Jun 06 22:15:16.671998 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.671974 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c984c879d-v6hc9"] Jun 06 22:15:16.676295 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:16.676274 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c984c879d-v6hc9"] Jun 06 22:15:17.490234 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:17.490193 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="742f19a7-ef72-4ba3-bc56-bc4e757bbe72" path="/var/lib/kubelet/pods/742f19a7-ef72-4ba3-bc56-bc4e757bbe72/volumes" Jun 06 22:15:36.486332 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.486297 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:36.547894 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.547858 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6897b7b5db-trrtt"] Jun 06 22:15:36.548270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.548256 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="742f19a7-ef72-4ba3-bc56-bc4e757bbe72" containerName="manager" Jun 06 22:15:36.548329 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.548271 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="742f19a7-ef72-4ba3-bc56-bc4e757bbe72" containerName="manager" Jun 06 22:15:36.548370 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.548363 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="742f19a7-ef72-4ba3-bc56-bc4e757bbe72" containerName="manager" Jun 06 22:15:36.552739 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.552721 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:36.555767 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.555745 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-w4pbt\"" Jun 06 22:15:36.555997 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.555981 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 06 22:15:36.566589 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.566563 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6897b7b5db-trrtt"] Jun 06 22:15:36.588140 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.588099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73c2cb19-a36f-4fdb-8595-eaab68a01b12-maas-api-tls\") pod \"maas-api-6897b7b5db-trrtt\" (UID: \"73c2cb19-a36f-4fdb-8595-eaab68a01b12\") " pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:36.588292 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.588164 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vlrd\" (UniqueName: \"kubernetes.io/projected/73c2cb19-a36f-4fdb-8595-eaab68a01b12-kube-api-access-7vlrd\") pod \"maas-api-6897b7b5db-trrtt\" (UID: \"73c2cb19-a36f-4fdb-8595-eaab68a01b12\") " pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:36.689656 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.689608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vlrd\" (UniqueName: \"kubernetes.io/projected/73c2cb19-a36f-4fdb-8595-eaab68a01b12-kube-api-access-7vlrd\") pod \"maas-api-6897b7b5db-trrtt\" (UID: \"73c2cb19-a36f-4fdb-8595-eaab68a01b12\") " pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:36.689905 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.689771 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73c2cb19-a36f-4fdb-8595-eaab68a01b12-maas-api-tls\") pod \"maas-api-6897b7b5db-trrtt\" (UID: \"73c2cb19-a36f-4fdb-8595-eaab68a01b12\") " pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:36.689997 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:15:36.689933 2580 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 06 22:15:36.690064 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:15:36.690013 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73c2cb19-a36f-4fdb-8595-eaab68a01b12-maas-api-tls podName:73c2cb19-a36f-4fdb-8595-eaab68a01b12 nodeName:}" failed. No retries permitted until 2026-06-06 22:15:37.189989911 +0000 UTC m=+764.188969373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/73c2cb19-a36f-4fdb-8595-eaab68a01b12-maas-api-tls") pod "maas-api-6897b7b5db-trrtt" (UID: "73c2cb19-a36f-4fdb-8595-eaab68a01b12") : secret "maas-api-serving-cert" not found Jun 06 22:15:36.698372 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.698340 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vlrd\" (UniqueName: \"kubernetes.io/projected/73c2cb19-a36f-4fdb-8595-eaab68a01b12-kube-api-access-7vlrd\") pod \"maas-api-6897b7b5db-trrtt\" (UID: \"73c2cb19-a36f-4fdb-8595-eaab68a01b12\") " pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:36.798964 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.798887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:36.922558 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:36.922518 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:37.021616 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.021586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:15:37.030540 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.030507 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-h7c4p"] Jun 06 22:15:37.038926 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.038901 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" Jun 06 22:15:37.042037 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.042003 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lcklf\"" Jun 06 22:15:37.042037 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.042003 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 06 22:15:37.050894 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.050834 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-h7c4p"] Jun 06 22:15:37.066413 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.066381 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-v7nzj"] Jun 06 22:15:37.070856 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.070835 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" Jun 06 22:15:37.081422 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.081399 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-v7nzj"] Jun 06 22:15:37.093171 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.093145 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5fp6\" (UniqueName: \"kubernetes.io/projected/2014ea99-c8c0-4bdc-906b-53670a0efcbc-kube-api-access-d5fp6\") pod \"payload-processing-7c6c859784-v7nzj\" (UID: \"2014ea99-c8c0-4bdc-906b-53670a0efcbc\") " pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" Jun 06 22:15:37.093336 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.093190 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm9mr\" (UniqueName: \"kubernetes.io/projected/3ce1b548-8661-433f-8a1a-401d468e7dcd-kube-api-access-wm9mr\") pod \"payload-pre-processing-67d7689b65-h7c4p\" (UID: \"3ce1b548-8661-433f-8a1a-401d468e7dcd\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" Jun 06 22:15:37.194024 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.193981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73c2cb19-a36f-4fdb-8595-eaab68a01b12-maas-api-tls\") pod \"maas-api-6897b7b5db-trrtt\" (UID: \"73c2cb19-a36f-4fdb-8595-eaab68a01b12\") " pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:37.194024 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.194028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm9mr\" (UniqueName: \"kubernetes.io/projected/3ce1b548-8661-433f-8a1a-401d468e7dcd-kube-api-access-wm9mr\") pod \"payload-pre-processing-67d7689b65-h7c4p\" (UID: \"3ce1b548-8661-433f-8a1a-401d468e7dcd\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" Jun 06 22:15:37.194293 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.194130 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5fp6\" (UniqueName: \"kubernetes.io/projected/2014ea99-c8c0-4bdc-906b-53670a0efcbc-kube-api-access-d5fp6\") pod \"payload-processing-7c6c859784-v7nzj\" (UID: \"2014ea99-c8c0-4bdc-906b-53670a0efcbc\") " pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" Jun 06 22:15:37.196372 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.196347 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73c2cb19-a36f-4fdb-8595-eaab68a01b12-maas-api-tls\") pod \"maas-api-6897b7b5db-trrtt\" (UID: \"73c2cb19-a36f-4fdb-8595-eaab68a01b12\") " pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:37.202666 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.202632 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm9mr\" (UniqueName: \"kubernetes.io/projected/3ce1b548-8661-433f-8a1a-401d468e7dcd-kube-api-access-wm9mr\") pod \"payload-pre-processing-67d7689b65-h7c4p\" (UID: \"3ce1b548-8661-433f-8a1a-401d468e7dcd\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" Jun 06 22:15:37.202765 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.202681 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5fp6\" (UniqueName: \"kubernetes.io/projected/2014ea99-c8c0-4bdc-906b-53670a0efcbc-kube-api-access-d5fp6\") pod \"payload-processing-7c6c859784-v7nzj\" (UID: \"2014ea99-c8c0-4bdc-906b-53670a0efcbc\") " pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" Jun 06 22:15:37.350612 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.350515 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" Jun 06 22:15:37.381484 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.381457 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" Jun 06 22:15:37.465731 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.465634 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:37.524246 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.524213 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-h7c4p"] Jun 06 22:15:37.525842 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:15:37.525779 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce1b548_8661_433f_8a1a_401d468e7dcd.slice/crio-8a965f7054342db5630cd492587b0a72b61751cbf07364fc86ad5ab421e2a4c7 WatchSource:0}: Error finding container 8a965f7054342db5630cd492587b0a72b61751cbf07364fc86ad5ab421e2a4c7: Status 404 returned error can't find the container with id 8a965f7054342db5630cd492587b0a72b61751cbf07364fc86ad5ab421e2a4c7 Jun 06 22:15:37.529203 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.528405 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 06 22:15:37.555790 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.555366 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-v7nzj"] Jun 06 22:15:37.669045 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.669019 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6897b7b5db-trrtt"] Jun 06 22:15:37.670955 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:15:37.670914 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c2cb19_a36f_4fdb_8595_eaab68a01b12.slice/crio-6584a01bf00f0877c80f98fb387f58554c2bf09125c61555aebba6930a2c6b74 WatchSource:0}: Error finding container 6584a01bf00f0877c80f98fb387f58554c2bf09125c61555aebba6930a2c6b74: Status 404 returned error can't find the container with id 6584a01bf00f0877c80f98fb387f58554c2bf09125c61555aebba6930a2c6b74 Jun 06 22:15:37.757047 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.757015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6897b7b5db-trrtt" event={"ID":"73c2cb19-a36f-4fdb-8595-eaab68a01b12","Type":"ContainerStarted","Data":"6584a01bf00f0877c80f98fb387f58554c2bf09125c61555aebba6930a2c6b74"} Jun 06 22:15:37.758269 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.758242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" event={"ID":"3ce1b548-8661-433f-8a1a-401d468e7dcd","Type":"ContainerStarted","Data":"8a965f7054342db5630cd492587b0a72b61751cbf07364fc86ad5ab421e2a4c7"} Jun 06 22:15:37.759273 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:37.759249 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" event={"ID":"2014ea99-c8c0-4bdc-906b-53670a0efcbc","Type":"ContainerStarted","Data":"0bdd9a94988d40253e0f16bb51e4f3825acb7cdc4911e7305e3790da96788208"} Jun 06 22:15:39.773933 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:39.773898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" event={"ID":"3ce1b548-8661-433f-8a1a-401d468e7dcd","Type":"ContainerStarted","Data":"17c5195c2b183ba5034ecf2c6330de255b52cb46fd419ae3bff2685c79eedbbc"} Jun 06 22:15:39.774480 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:39.774153 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" Jun 06 22:15:39.775618 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:39.775579 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" event={"ID":"2014ea99-c8c0-4bdc-906b-53670a0efcbc","Type":"ContainerStarted","Data":"450d50167a4f9df9ec130420ff17483832de5c31a7929b0cd3771d0eb0f40c8f"} Jun 06 22:15:39.775736 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:39.775636 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" Jun 06 22:15:39.777257 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:39.777229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6897b7b5db-trrtt" event={"ID":"73c2cb19-a36f-4fdb-8595-eaab68a01b12","Type":"ContainerStarted","Data":"b3d8584eea47f7c237925886bdbb7f00508103321e873ca033d32ed4ff2ad223"} Jun 06 22:15:39.777364 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:39.777287 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:39.788755 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:39.788702 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" podStartSLOduration=0.682821293 podStartE2EDuration="2.788685968s" totalImagesPullingTime="2.105864675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:15:39.788137817 +0000 UTC m=+766.787117304" watchObservedRunningTime="2026-06-06 22:15:39.788685968 +0000 UTC m=+766.787665458" Jun 06 22:15:39.803496 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:39.803411 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" podStartSLOduration=0.73345504 podStartE2EDuration="2.803394571s" totalImagesPullingTime="2.069939531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:15:39.802369119 +0000 UTC m=+766.801348603" watchObservedRunningTime="2026-06-06 22:15:39.803394571 +0000 UTC m=+766.802374037" Jun 06 22:15:39.822439 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:39.822372 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6897b7b5db-trrtt" podStartSLOduration=1.8628219050000001 podStartE2EDuration="3.822349622s" totalImagesPullingTime="1.959527717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:15:39.820692923 +0000 UTC m=+766.819672411" watchObservedRunningTime="2026-06-06 22:15:39.822349622 +0000 UTC m=+766.821329110" Jun 06 22:15:44.803281 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:44.803248 2580 generic.go:356] "Generic (PLEG): container finished" podID="aeb14052-2c78-4059-9b3d-4d0c9bec4f4e" containerID="4f027a554683f891ba35569813d96beec8abf9ca34fdef88786abf30ea89ec64" exitCode=0 Jun 06 22:15:44.803694 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:44.803321 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" event={"ID":"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e","Type":"ContainerDied","Data":"4f027a554683f891ba35569813d96beec8abf9ca34fdef88786abf30ea89ec64"} Jun 06 22:15:44.803736 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:44.803699 2580 scope.go:117] "RemoveContainer" containerID="4f027a554683f891ba35569813d96beec8abf9ca34fdef88786abf30ea89ec64" Jun 06 22:15:44.804665 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:44.804637 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:15:44.804733 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:44.804708 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:15:44.804773 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:44.804740 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:15:45.787747 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:45.787718 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6897b7b5db-trrtt" Jun 06 22:15:45.822831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:45.816792 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" event={"ID":"aeb14052-2c78-4059-9b3d-4d0c9bec4f4e","Type":"ContainerStarted","Data":"c1bebb3a12cecb55a00e0bd72fd4057515c4b18fb53b49edbae6c37628f80dc3"} Jun 06 22:15:46.461266 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:46.461225 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:15:46.462507 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:46.462484 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-zrt7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jun 06 22:15:46.462576 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:46.462533 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" podUID="aeb14052-2c78-4059-9b3d-4d0c9bec4f4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jun 06 22:15:47.461507 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:47.461469 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-zrt7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jun 06 22:15:47.462035 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:47.461528 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" podUID="aeb14052-2c78-4059-9b3d-4d0c9bec4f4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jun 06 22:15:47.943502 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:47.943468 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 06 22:15:47.943751 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:47.943661 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" containerID="cri-o://9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392" gracePeriod=30 Jun 06 22:15:48.461499 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:48.461465 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-zrt7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jun 06 22:15:48.461676 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:48.461514 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" podUID="aeb14052-2c78-4059-9b3d-4d0c9bec4f4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jun 06 22:15:49.003197 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.003153 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:15:49.461662 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.461394 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-zrt7w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jun 06 22:15:49.461662 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.461460 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" podUID="aeb14052-2c78-4059-9b3d-4d0c9bec4f4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jun 06 22:15:49.609692 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.609665 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:49.646693 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.646665 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp9dx\" (UniqueName: \"kubernetes.io/projected/5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36-kube-api-access-gp9dx\") pod \"5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36\" (UID: \"5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36\") " Jun 06 22:15:49.649108 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.649071 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36-kube-api-access-gp9dx" (OuterVolumeSpecName: "kube-api-access-gp9dx") pod "5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" (UID: "5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36"). InnerVolumeSpecName "kube-api-access-gp9dx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:15:49.747785 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.747741 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp9dx\" (UniqueName: \"kubernetes.io/projected/5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36-kube-api-access-gp9dx\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:15:49.839057 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.839022 2580 generic.go:356] "Generic (PLEG): container finished" podID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerID="9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392" exitCode=143 Jun 06 22:15:49.839234 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.839071 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:49.839234 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.839094 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36","Type":"ContainerDied","Data":"9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392"} Jun 06 22:15:49.839234 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.839149 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36","Type":"ContainerDied","Data":"35f9a245c62e85031661c8527683d15c6c7a43856ac67b25f933014e69812853"} Jun 06 22:15:49.839234 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.839172 2580 scope.go:117] "RemoveContainer" containerID="9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392" Jun 06 22:15:49.849336 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.849314 2580 scope.go:117] "RemoveContainer" containerID="9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392" Jun 06 22:15:49.849595 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:15:49.849576 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392\": container with ID starting with 9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392 not found: ID does not exist" containerID="9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392" Jun 06 22:15:49.849672 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.849609 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392"} err="failed to get container status \"9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392\": rpc error: code = NotFound desc = could not find container \"9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392\": container with ID starting with 9dc3014d5ac46461e239307cdb294ab40c2873ed9095c258e31f370bfe1ff392 not found: ID does not exist" Jun 06 22:15:49.860657 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.860630 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 06 22:15:49.867936 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.863873 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 06 22:15:49.884362 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.884318 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 06 22:15:49.884933 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.884910 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" Jun 06 22:15:49.885020 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.884936 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" Jun 06 22:15:49.885073 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.885032 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" containerName="keycloak" Jun 06 22:15:49.889422 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.889405 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:49.892084 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.892061 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 06 22:15:49.892084 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.892062 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 06 22:15:49.892270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.892062 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-4d87f\"" Jun 06 22:15:49.892270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.892067 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 06 22:15:49.892270 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.892105 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 06 22:15:49.896228 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.896204 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 06 22:15:49.949900 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.949785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a-test-realms\") pod \"maas-keycloak-0\" (UID: \"9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a\") " pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:49.949900 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:49.949849 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhzqr\" (UniqueName: \"kubernetes.io/projected/9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a-kube-api-access-zhzqr\") pod \"maas-keycloak-0\" (UID: \"9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a\") " pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:50.051171 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.051126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a-test-realms\") pod \"maas-keycloak-0\" (UID: \"9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a\") " pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:50.051340 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.051187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhzqr\" (UniqueName: \"kubernetes.io/projected/9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a-kube-api-access-zhzqr\") pod \"maas-keycloak-0\" (UID: \"9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a\") " pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:50.051944 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.051921 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a-test-realms\") pod \"maas-keycloak-0\" (UID: \"9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a\") " pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:50.059644 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.059611 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhzqr\" (UniqueName: \"kubernetes.io/projected/9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a-kube-api-access-zhzqr\") pod \"maas-keycloak-0\" (UID: \"9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a\") " pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:50.200395 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.200293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:50.341428 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.341400 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 06 22:15:50.343702 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:15:50.343674 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3ffba1_cc36_44bb_9dd6_0156e6e8c68a.slice/crio-2c7bf6969454c3e6b03fcca96cd8be42f0860e0a189f5208e5d0c29cb1336c0c WatchSource:0}: Error finding container 2c7bf6969454c3e6b03fcca96cd8be42f0860e0a189f5208e5d0c29cb1336c0c: Status 404 returned error can't find the container with id 2c7bf6969454c3e6b03fcca96cd8be42f0860e0a189f5208e5d0c29cb1336c0c Jun 06 22:15:50.465298 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.465217 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:15:50.466157 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.466136 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-zrt7w" Jun 06 22:15:50.784952 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.784867 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-h7c4p" Jun 06 22:15:50.784952 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.784940 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-v7nzj" Jun 06 22:15:50.847692 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.847650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a","Type":"ContainerStarted","Data":"f63e3854303348383f3fc1c6550f9c4818c3ce88fe46f265357cf791e09b6c61"} Jun 06 22:15:50.847978 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.847952 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a","Type":"ContainerStarted","Data":"2c7bf6969454c3e6b03fcca96cd8be42f0860e0a189f5208e5d0c29cb1336c0c"} Jun 06 22:15:50.865859 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:50.865787 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.551201146 podStartE2EDuration="1.86576647s" totalImagesPullingTime="314.565324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:15:50.864105431 +0000 UTC m=+777.863084917" watchObservedRunningTime="2026-06-06 22:15:50.86576647 +0000 UTC m=+777.864745956" Jun 06 22:15:51.201159 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:51.201109 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 06 22:15:51.202969 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:51.202933 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:15:51.491011 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:51.490918 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36" path="/var/lib/kubelet/pods/5a2491ce-b6ae-4c63-bfcd-333dcf7f1c36/volumes" Jun 06 22:15:52.201747 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:52.201697 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:15:53.201469 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:53.201402 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:15:54.201214 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:54.201152 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:15:55.201595 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:55.201533 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:15:56.202073 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:56.201598 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:15:57.201267 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:57.201218 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:15:58.201774 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:58.201732 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:15:59.202276 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:15:59.202218 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:16:00.201091 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:00.201042 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 06 22:16:00.201452 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:00.201412 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:16:01.201089 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:01.201034 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:16:02.201424 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:02.201374 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:16:03.201505 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:03.201460 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 06 22:16:04.319145 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:04.319102 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 06 22:16:04.339824 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:04.339756 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:16:14.324380 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:14.324270 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="9d3ffba1-cc36-44bb-9dd6-0156e6e8c68a" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 06 22:16:24.325239 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:24.325207 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 06 22:16:36.955938 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:36.955877 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:16:37.045321 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:37.045270 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:16:48.055401 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.055363 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l"] Jun 06 22:16:48.058685 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.058663 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.062763 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.062737 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 06 22:16:48.063018 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.062815 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 06 22:16:48.063135 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.062817 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mh76x\"" Jun 06 22:16:48.063256 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.063237 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 06 22:16:48.069530 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.069497 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l"] Jun 06 22:16:48.087105 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.087078 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:16:48.237017 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.236985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.237217 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.237086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r8jc\" (UniqueName: \"kubernetes.io/projected/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-kube-api-access-2r8jc\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.237217 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.237115 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.237217 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.237168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.237392 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.237235 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.237392 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.237283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.338454 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.338356 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.338454 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.338422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.338683 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.338477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.338683 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.338510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.338683 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.338593 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8jc\" (UniqueName: \"kubernetes.io/projected/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-kube-api-access-2r8jc\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.338683 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.338617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.338934 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.338767 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.338934 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.338853 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.338934 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.338898 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.340767 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.340744 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.341160 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.341143 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.347433 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.347409 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r8jc\" (UniqueName: \"kubernetes.io/projected/4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e-kube-api-access-2r8jc\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l\" (UID: \"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.371134 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.371113 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:48.509218 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:48.509186 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l"] Jun 06 22:16:48.512059 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:16:48.512026 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a647be8_4e2c_4ae3_baf4_dbe0ce5a9f8e.slice/crio-205392cf277b0c7ca14eff69d87dd35a7bfcaeddf828c4c61fdccc6540618f57 WatchSource:0}: Error finding container 205392cf277b0c7ca14eff69d87dd35a7bfcaeddf828c4c61fdccc6540618f57: Status 404 returned error can't find the container with id 205392cf277b0c7ca14eff69d87dd35a7bfcaeddf828c4c61fdccc6540618f57 Jun 06 22:16:49.200954 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:49.200912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" event={"ID":"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e","Type":"ContainerStarted","Data":"205392cf277b0c7ca14eff69d87dd35a7bfcaeddf828c4c61fdccc6540618f57"} Jun 06 22:16:53.228117 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:53.228070 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" event={"ID":"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e","Type":"ContainerStarted","Data":"6c0ddcdf4fa763664ee8bf063c01981c143a8c682db466338d6bd51b7e4689ce"} Jun 06 22:16:54.234353 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:54.234316 2580 generic.go:356] "Generic (PLEG): container finished" podID="4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e" containerID="6c0ddcdf4fa763664ee8bf063c01981c143a8c682db466338d6bd51b7e4689ce" exitCode=0 Jun 06 22:16:54.234797 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:54.234362 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" event={"ID":"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e","Type":"ContainerDied","Data":"6c0ddcdf4fa763664ee8bf063c01981c143a8c682db466338d6bd51b7e4689ce"} Jun 06 22:16:56.246173 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:56.246119 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" event={"ID":"4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e","Type":"ContainerStarted","Data":"ae3feddc405a0d06a16333cd801e2b78c7967b69ee6c2095325984275d034ccf"} Jun 06 22:16:56.265869 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:56.265816 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" podStartSLOduration=2.258394514 podStartE2EDuration="8.265783287s" totalImagesPullingTime="4.753343752s" totalInitContainerRuntime="1.254045021s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:16:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:16:56.263742271 +0000 UTC m=+843.262721757" watchObservedRunningTime="2026-06-06 22:16:56.265783287 +0000 UTC m=+843.264762772" Jun 06 22:16:58.372261 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:58.372213 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:58.372719 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:58.372286 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:58.384770 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:58.384745 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:58.975146 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:58.975107 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:16:59.274920 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:59.274828 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l" Jun 06 22:16:59.982734 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:16:59.982702 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:17:01.364631 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.364596 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f"] Jun 06 22:17:01.371598 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.371575 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.374489 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.374464 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 06 22:17:01.378723 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.378700 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f"] Jun 06 22:17:01.394502 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.394460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:17:01.463518 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.463482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgl6m\" (UniqueName: \"kubernetes.io/projected/d0d7b6af-c5d3-4841-aeff-74ae805ae528-kube-api-access-hgl6m\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.463518 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.463523 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.463785 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.463556 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0d7b6af-c5d3-4841-aeff-74ae805ae528-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.463785 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.463662 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.463785 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.463738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.463973 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.463791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.564934 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.564886 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0d7b6af-c5d3-4841-aeff-74ae805ae528-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.565204 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.565178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.565278 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.565265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.565339 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.565310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.565420 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.565398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgl6m\" (UniqueName: \"kubernetes.io/projected/d0d7b6af-c5d3-4841-aeff-74ae805ae528-kube-api-access-hgl6m\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.565483 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.565430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.565642 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.565617 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.565730 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.565626 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.565730 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.565707 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.567518 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.567492 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0d7b6af-c5d3-4841-aeff-74ae805ae528-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.567615 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.567596 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0d7b6af-c5d3-4841-aeff-74ae805ae528-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.573453 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.573427 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgl6m\" (UniqueName: \"kubernetes.io/projected/d0d7b6af-c5d3-4841-aeff-74ae805ae528-kube-api-access-hgl6m\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f\" (UID: \"d0d7b6af-c5d3-4841-aeff-74ae805ae528\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.683600 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.683570 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:01.828282 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:01.828256 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f"] Jun 06 22:17:01.830380 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:17:01.830347 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d7b6af_c5d3_4841_aeff_74ae805ae528.slice/crio-410ead7c03e3df19852d89cebf013904f78bfdc6402cafeb82055540a1cdeaa6 WatchSource:0}: Error finding container 410ead7c03e3df19852d89cebf013904f78bfdc6402cafeb82055540a1cdeaa6: Status 404 returned error can't find the container with id 410ead7c03e3df19852d89cebf013904f78bfdc6402cafeb82055540a1cdeaa6 Jun 06 22:17:02.282896 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:02.282860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" event={"ID":"d0d7b6af-c5d3-4841-aeff-74ae805ae528","Type":"ContainerStarted","Data":"16bfc35d0260f640c322dd49e5cfddbeb99c40ee6d79fd4795de34d3ff6b6d70"} Jun 06 22:17:02.283090 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:02.282912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" event={"ID":"d0d7b6af-c5d3-4841-aeff-74ae805ae528","Type":"ContainerStarted","Data":"410ead7c03e3df19852d89cebf013904f78bfdc6402cafeb82055540a1cdeaa6"} Jun 06 22:17:03.289159 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:03.289128 2580 generic.go:356] "Generic (PLEG): container finished" podID="d0d7b6af-c5d3-4841-aeff-74ae805ae528" containerID="16bfc35d0260f640c322dd49e5cfddbeb99c40ee6d79fd4795de34d3ff6b6d70" exitCode=0 Jun 06 22:17:03.289522 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:03.289210 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" event={"ID":"d0d7b6af-c5d3-4841-aeff-74ae805ae528","Type":"ContainerDied","Data":"16bfc35d0260f640c322dd49e5cfddbeb99c40ee6d79fd4795de34d3ff6b6d70"} Jun 06 22:17:04.296169 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:04.296127 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" event={"ID":"d0d7b6af-c5d3-4841-aeff-74ae805ae528","Type":"ContainerStarted","Data":"f637853fbca8df2bb4ac3840ed13e6f1ecac7478d8ba9185b34aa5c4370d6232"} Jun 06 22:17:04.315595 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:04.315541 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" podStartSLOduration=1.783502828 podStartE2EDuration="3.31552467s" totalImagesPullingTime="177.429293ms" totalInitContainerRuntime="1.354592549s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:17:04.314358146 +0000 UTC m=+851.313337631" watchObservedRunningTime="2026-06-06 22:17:04.31552467 +0000 UTC m=+851.314504154" Jun 06 22:17:05.757951 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:05.757918 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:17:05.810508 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:05.810469 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:17:11.683869 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:11.683779 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:11.684298 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:11.683883 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:11.696406 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:11.696380 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:12.349406 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:12.349378 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f" Jun 06 22:17:22.266971 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.266940 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n"] Jun 06 22:17:22.270860 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.270840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.273463 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.273441 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 06 22:17:22.279855 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.279831 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n"] Jun 06 22:17:22.290109 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.290082 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:17:22.372261 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.372227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40569393-55f7-4d74-836a-593f190d22b1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.372431 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.372277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.372431 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.372299 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm6qd\" (UniqueName: \"kubernetes.io/projected/40569393-55f7-4d74-836a-593f190d22b1-kube-api-access-mm6qd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.372431 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.372409 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.372543 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.372464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.372580 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.372540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.473714 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.473671 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.473943 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.473731 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.473943 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.473839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.473943 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.473877 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40569393-55f7-4d74-836a-593f190d22b1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.473943 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.473925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.474188 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.473953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm6qd\" (UniqueName: \"kubernetes.io/projected/40569393-55f7-4d74-836a-593f190d22b1-kube-api-access-mm6qd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.474247 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.474194 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.474302 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.474255 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.474345 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.474315 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.476234 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.476206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40569393-55f7-4d74-836a-593f190d22b1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.476616 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.476594 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40569393-55f7-4d74-836a-593f190d22b1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.482092 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.482068 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm6qd\" (UniqueName: \"kubernetes.io/projected/40569393-55f7-4d74-836a-593f190d22b1-kube-api-access-mm6qd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n\" (UID: \"40569393-55f7-4d74-836a-593f190d22b1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.583168 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.583078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:22.733025 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:22.732993 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n"] Jun 06 22:17:22.734643 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:17:22.734614 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40569393_55f7_4d74_836a_593f190d22b1.slice/crio-fce68e9262422012679e5814ba767bad8879fb8801acf02691d8c85ce25744ac WatchSource:0}: Error finding container fce68e9262422012679e5814ba767bad8879fb8801acf02691d8c85ce25744ac: Status 404 returned error can't find the container with id fce68e9262422012679e5814ba767bad8879fb8801acf02691d8c85ce25744ac Jun 06 22:17:23.393735 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:23.393696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" event={"ID":"40569393-55f7-4d74-836a-593f190d22b1","Type":"ContainerStarted","Data":"0b803bb7867de711df0b6cba5e4ee345d24270e102011d923686df0fdc0030a1"} Jun 06 22:17:23.394272 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:23.393764 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" event={"ID":"40569393-55f7-4d74-836a-593f190d22b1","Type":"ContainerStarted","Data":"fce68e9262422012679e5814ba767bad8879fb8801acf02691d8c85ce25744ac"} Jun 06 22:17:24.400999 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:24.400960 2580 generic.go:356] "Generic (PLEG): container finished" podID="40569393-55f7-4d74-836a-593f190d22b1" containerID="0b803bb7867de711df0b6cba5e4ee345d24270e102011d923686df0fdc0030a1" exitCode=0 Jun 06 22:17:24.401436 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:24.401046 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" event={"ID":"40569393-55f7-4d74-836a-593f190d22b1","Type":"ContainerDied","Data":"0b803bb7867de711df0b6cba5e4ee345d24270e102011d923686df0fdc0030a1"} Jun 06 22:17:25.408445 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:25.408407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" event={"ID":"40569393-55f7-4d74-836a-593f190d22b1","Type":"ContainerStarted","Data":"713cbc5eece5a2af40b3b318e2c3d52b50ec2e68d8a6de7f350d42dc410e5437"} Jun 06 22:17:25.426896 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:25.426841 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" podStartSLOduration=1.944537542 podStartE2EDuration="3.426822917s" totalImagesPullingTime="179.989208ms" totalInitContainerRuntime="1.302296167s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:17:25.424988589 +0000 UTC m=+872.423968075" watchObservedRunningTime="2026-06-06 22:17:25.426822917 +0000 UTC m=+872.425802399" Jun 06 22:17:32.583850 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:32.583796 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:32.584288 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:32.583883 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:32.597421 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:32.597394 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:33.461745 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:33.461716 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n" Jun 06 22:17:34.150467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.150397 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8"] Jun 06 22:17:34.155732 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.155708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.158262 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.158242 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 06 22:17:34.168321 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.168295 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8"] Jun 06 22:17:34.175916 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.175887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:17:34.192467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.192435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.192467 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.192468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.192711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.192507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.192711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.192551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.192711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.192573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf8z7\" (UniqueName: \"kubernetes.io/projected/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-kube-api-access-kf8z7\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.192711 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.192648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.294076 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.294040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.294259 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.294083 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.294259 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.294112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf8z7\" (UniqueName: \"kubernetes.io/projected/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-kube-api-access-kf8z7\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.294259 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.294184 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.294428 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.294267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.294428 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.294289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.294528 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.294502 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.294622 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.294598 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.294690 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.294645 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.296582 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.296553 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.297026 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.297007 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.302282 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.302251 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf8z7\" (UniqueName: \"kubernetes.io/projected/d3c8ae5d-0217-4b11-8f1d-e202391ed03e-kube-api-access-kf8z7\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-glsx8\" (UID: \"d3c8ae5d-0217-4b11-8f1d-e202391ed03e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.467692 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.467611 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:34.608378 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:34.608341 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8"] Jun 06 22:17:34.610533 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:17:34.610504 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c8ae5d_0217_4b11_8f1d_e202391ed03e.slice/crio-8aaa561633f943c94b43664f912524c843ad738fab1f0d61affbafec158c26f5 WatchSource:0}: Error finding container 8aaa561633f943c94b43664f912524c843ad738fab1f0d61affbafec158c26f5: Status 404 returned error can't find the container with id 8aaa561633f943c94b43664f912524c843ad738fab1f0d61affbafec158c26f5 Jun 06 22:17:35.462067 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:35.462031 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" event={"ID":"d3c8ae5d-0217-4b11-8f1d-e202391ed03e","Type":"ContainerStarted","Data":"3bc16a6b62f7fddade1ba025f8a52f4561e5c4cfd3b7db66a213037569e8dc49"} Jun 06 22:17:35.462550 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:35.462090 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" event={"ID":"d3c8ae5d-0217-4b11-8f1d-e202391ed03e","Type":"ContainerStarted","Data":"8aaa561633f943c94b43664f912524c843ad738fab1f0d61affbafec158c26f5"} Jun 06 22:17:36.468853 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:36.468823 2580 generic.go:356] "Generic (PLEG): container finished" podID="d3c8ae5d-0217-4b11-8f1d-e202391ed03e" containerID="3bc16a6b62f7fddade1ba025f8a52f4561e5c4cfd3b7db66a213037569e8dc49" exitCode=0 Jun 06 22:17:36.469255 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:36.468877 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" event={"ID":"d3c8ae5d-0217-4b11-8f1d-e202391ed03e","Type":"ContainerDied","Data":"3bc16a6b62f7fddade1ba025f8a52f4561e5c4cfd3b7db66a213037569e8dc49"} Jun 06 22:17:37.477228 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:37.477177 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" event={"ID":"d3c8ae5d-0217-4b11-8f1d-e202391ed03e","Type":"ContainerStarted","Data":"5513200d76171aeadfe312fd6253219fe8c7840c3cfc298c44ac7c9d43d750c1"} Jun 06 22:17:37.495378 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:37.495324 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" podStartSLOduration=2.050238278 podStartE2EDuration="3.495308063s" totalImagesPullingTime="173.471738ms" totalInitContainerRuntime="1.271598047s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:17:37.494485989 +0000 UTC m=+884.493465474" watchObservedRunningTime="2026-06-06 22:17:37.495308063 +0000 UTC m=+884.494287562" Jun 06 22:17:41.106356 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:41.106322 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:17:44.468547 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:44.468461 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:44.469078 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:44.468552 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:44.481202 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:44.481179 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:44.526011 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:44.525971 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-glsx8" Jun 06 22:17:53.507551 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:53.507514 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:17:53.509014 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:53.508992 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:17:53.514100 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:53.514079 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:17:53.515129 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:17:53.515108 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:18:11.289151 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:18:11.289109 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:18:11.390128 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:18:11.390087 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:18:33.121940 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:18:33.121905 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:18:33.220977 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:18:33.220945 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:18:34.186226 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:18:34.186188 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:18:34.282379 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:18:34.282344 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:20:01.334667 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.334625 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:20:01.437571 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.437531 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-844f958585-6dchl"] Jun 06 22:20:01.437795 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.437755 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-844f958585-6dchl" podUID="496c6e85-8cb6-4100-9f0b-408d4f3a75eb" containerName="manager" containerID="cri-o://3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4" gracePeriod=30 Jun 06 22:20:01.698085 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.698059 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:20:01.773061 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.773032 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-webhook-cert\") pod \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\" (UID: \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\") " Jun 06 22:20:01.773257 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.773083 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8784n\" (UniqueName: \"kubernetes.io/projected/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-kube-api-access-8784n\") pod \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\" (UID: \"496c6e85-8cb6-4100-9f0b-408d4f3a75eb\") " Jun 06 22:20:01.775205 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.775172 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-kube-api-access-8784n" (OuterVolumeSpecName: "kube-api-access-8784n") pod "496c6e85-8cb6-4100-9f0b-408d4f3a75eb" (UID: "496c6e85-8cb6-4100-9f0b-408d4f3a75eb"). InnerVolumeSpecName "kube-api-access-8784n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:20:01.775325 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.775253 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "496c6e85-8cb6-4100-9f0b-408d4f3a75eb" (UID: "496c6e85-8cb6-4100-9f0b-408d4f3a75eb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 06 22:20:01.874661 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.874572 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-webhook-cert\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:20:01.874661 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:01.874601 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8784n\" (UniqueName: \"kubernetes.io/projected/496c6e85-8cb6-4100-9f0b-408d4f3a75eb-kube-api-access-8784n\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:20:02.263752 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.263716 2580 generic.go:356] "Generic (PLEG): container finished" podID="496c6e85-8cb6-4100-9f0b-408d4f3a75eb" containerID="3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4" exitCode=0 Jun 06 22:20:02.263947 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.263764 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844f958585-6dchl" Jun 06 22:20:02.263947 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.263827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844f958585-6dchl" event={"ID":"496c6e85-8cb6-4100-9f0b-408d4f3a75eb","Type":"ContainerDied","Data":"3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4"} Jun 06 22:20:02.263947 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.263890 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844f958585-6dchl" event={"ID":"496c6e85-8cb6-4100-9f0b-408d4f3a75eb","Type":"ContainerDied","Data":"a4b268ae635124fa7e086b0d46095aa5e6b157b21b33f5f43c1c1bec4dd76ede"} Jun 06 22:20:02.263947 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.263915 2580 scope.go:117] "RemoveContainer" containerID="3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4" Jun 06 22:20:02.273558 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.273539 2580 scope.go:117] "RemoveContainer" containerID="3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4" Jun 06 22:20:02.273827 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:20:02.273790 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4\": container with ID starting with 3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4 not found: ID does not exist" containerID="3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4" Jun 06 22:20:02.273907 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.273835 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4"} err="failed to get container status \"3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4\": rpc error: code = NotFound desc = could not find container \"3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4\": container with ID starting with 3cf6fdbafeda12648047c8c7a5f6653a8ae84afd6a5df805bf477e1616051cd4 not found: ID does not exist" Jun 06 22:20:02.284044 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.284007 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-844f958585-6dchl"] Jun 06 22:20:02.287300 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.287277 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-844f958585-6dchl"] Jun 06 22:20:02.912616 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:02.912581 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:20:03.015718 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.015682 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-844f958585-g5bsk"] Jun 06 22:20:03.016168 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.016154 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="496c6e85-8cb6-4100-9f0b-408d4f3a75eb" containerName="manager" Jun 06 22:20:03.016223 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.016170 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="496c6e85-8cb6-4100-9f0b-408d4f3a75eb" containerName="manager" Jun 06 22:20:03.016306 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.016241 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="496c6e85-8cb6-4100-9f0b-408d4f3a75eb" containerName="manager" Jun 06 22:20:03.020602 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.020583 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:03.023516 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.023496 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 06 22:20:03.024501 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.024474 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wgdtv\"" Jun 06 22:20:03.024626 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.024487 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 06 22:20:03.026171 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.026102 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-844f958585-g5bsk"] Jun 06 22:20:03.088489 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.088457 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f9ce1cb-eca3-480f-8cad-c1cd57590f63-webhook-cert\") pod \"maas-controller-844f958585-g5bsk\" (UID: \"2f9ce1cb-eca3-480f-8cad-c1cd57590f63\") " pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:03.088682 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.088537 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nfkk\" (UniqueName: \"kubernetes.io/projected/2f9ce1cb-eca3-480f-8cad-c1cd57590f63-kube-api-access-7nfkk\") pod \"maas-controller-844f958585-g5bsk\" (UID: \"2f9ce1cb-eca3-480f-8cad-c1cd57590f63\") " pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:03.189294 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.189202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f9ce1cb-eca3-480f-8cad-c1cd57590f63-webhook-cert\") pod \"maas-controller-844f958585-g5bsk\" (UID: \"2f9ce1cb-eca3-480f-8cad-c1cd57590f63\") " pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:03.189294 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.189273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nfkk\" (UniqueName: \"kubernetes.io/projected/2f9ce1cb-eca3-480f-8cad-c1cd57590f63-kube-api-access-7nfkk\") pod \"maas-controller-844f958585-g5bsk\" (UID: \"2f9ce1cb-eca3-480f-8cad-c1cd57590f63\") " pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:03.191721 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.191695 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f9ce1cb-eca3-480f-8cad-c1cd57590f63-webhook-cert\") pod \"maas-controller-844f958585-g5bsk\" (UID: \"2f9ce1cb-eca3-480f-8cad-c1cd57590f63\") " pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:03.196848 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.196798 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nfkk\" (UniqueName: \"kubernetes.io/projected/2f9ce1cb-eca3-480f-8cad-c1cd57590f63-kube-api-access-7nfkk\") pod \"maas-controller-844f958585-g5bsk\" (UID: \"2f9ce1cb-eca3-480f-8cad-c1cd57590f63\") " pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:03.335679 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.335641 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:03.471867 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.471839 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-844f958585-g5bsk"] Jun 06 22:20:03.474018 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:20:03.473979 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f9ce1cb_eca3_480f_8cad_c1cd57590f63.slice/crio-85601493a368951dfeb73b5895accd4d50a18c25e8b636dc9d2de11d7b65dc06 WatchSource:0}: Error finding container 85601493a368951dfeb73b5895accd4d50a18c25e8b636dc9d2de11d7b65dc06: Status 404 returned error can't find the container with id 85601493a368951dfeb73b5895accd4d50a18c25e8b636dc9d2de11d7b65dc06 Jun 06 22:20:03.490590 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:03.490565 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="496c6e85-8cb6-4100-9f0b-408d4f3a75eb" path="/var/lib/kubelet/pods/496c6e85-8cb6-4100-9f0b-408d4f3a75eb/volumes" Jun 06 22:20:04.278112 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:04.278072 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844f958585-g5bsk" event={"ID":"2f9ce1cb-eca3-480f-8cad-c1cd57590f63","Type":"ContainerStarted","Data":"397c242af82b77116b89933ac760614afb416ff596d3d4f495148594585585fb"} Jun 06 22:20:04.278545 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:04.278133 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844f958585-g5bsk" event={"ID":"2f9ce1cb-eca3-480f-8cad-c1cd57590f63","Type":"ContainerStarted","Data":"85601493a368951dfeb73b5895accd4d50a18c25e8b636dc9d2de11d7b65dc06"} Jun 06 22:20:04.278545 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:04.278163 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:04.293221 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:04.293172 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-844f958585-g5bsk" podStartSLOduration=0.917522814 podStartE2EDuration="1.293156067s" totalImagesPullingTime="375.633253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:20:04.292128849 +0000 UTC m=+1031.291108333" watchObservedRunningTime="2026-06-06 22:20:04.293156067 +0000 UTC m=+1031.292135550" Jun 06 22:20:15.288896 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:15.288864 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-844f958585-g5bsk" Jun 06 22:20:42.739934 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:42.739901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:20:42.838082 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:42.838001 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:20:51.203036 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:51.203001 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:20:57.005788 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:57.005759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:20:57.068389 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:20:57.068359 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:21:29.125097 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:21:29.125062 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:21:29.224973 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:21:29.224937 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:21:53.093925 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:21:53.093893 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:21:59.112599 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:21:59.112564 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:21:59.213731 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:21:59.213684 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:21:59.317280 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:21:59.317241 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:21:59.456371 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:21:59.456336 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:21:59.556929 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:21:59.556899 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:15.871731 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:15.871644 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:16.876054 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:16.876014 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:21.802659 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:21.802622 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:21.859447 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:21.859411 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:29.935614 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:29.935572 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:30.030480 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:30.030446 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:30.131258 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:30.131226 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:30.230602 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:30.230496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:38.053854 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:38.053822 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:41.030915 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:41.030885 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:46.562601 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:46.562566 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:46.664958 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:46.664925 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:46.773870 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:46.773834 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:52.565857 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:52.565782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:52.624237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:52.624206 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:22:53.540374 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:53.540351 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:22:53.543854 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:53.543832 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:22:53.547539 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:53.547517 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:22:53.550486 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:22:53.550468 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:23:12.786726 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:23:12.786682 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:23:12.884023 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:23:12.883992 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:23:12.983867 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:23:12.983831 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:23:18.552216 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:23:18.552172 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:23:18.652901 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:23:18.652868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:23:18.753562 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:23:18.753529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:23:18.871381 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:23:18.871291 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:23:24.442477 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:23:24.442443 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:23:24.541053 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:23:24.541019 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:26:28.093043 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:26:28.093011 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:26:28.143204 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:26:28.143169 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:27:53.574734 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:27:53.574702 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:27:53.579127 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:27:53.579099 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:27:53.581386 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:27:53.581366 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:27:53.585506 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:27:53.585489 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:29:05.159180 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:29:05.159145 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:30:24.984505 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:24.984466 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9"] Jun 06 22:30:24.984986 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:24.984658 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" podUID="7734961d-c56c-4b7f-a456-42a70e164474" containerName="manager" containerID="cri-o://3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36" gracePeriod=10 Jun 06 22:30:25.248087 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.248029 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:30:25.334939 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.334909 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lw2d\" (UniqueName: \"kubernetes.io/projected/7734961d-c56c-4b7f-a456-42a70e164474-kube-api-access-9lw2d\") pod \"7734961d-c56c-4b7f-a456-42a70e164474\" (UID: \"7734961d-c56c-4b7f-a456-42a70e164474\") " Jun 06 22:30:25.335098 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.335042 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7734961d-c56c-4b7f-a456-42a70e164474-extensions-socket-volume\") pod \"7734961d-c56c-4b7f-a456-42a70e164474\" (UID: \"7734961d-c56c-4b7f-a456-42a70e164474\") " Jun 06 22:30:25.335396 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.335371 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7734961d-c56c-4b7f-a456-42a70e164474-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7734961d-c56c-4b7f-a456-42a70e164474" (UID: "7734961d-c56c-4b7f-a456-42a70e164474"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:30:25.336908 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.336885 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7734961d-c56c-4b7f-a456-42a70e164474-kube-api-access-9lw2d" (OuterVolumeSpecName: "kube-api-access-9lw2d") pod "7734961d-c56c-4b7f-a456-42a70e164474" (UID: "7734961d-c56c-4b7f-a456-42a70e164474"). InnerVolumeSpecName "kube-api-access-9lw2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:30:25.436554 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.436520 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lw2d\" (UniqueName: \"kubernetes.io/projected/7734961d-c56c-4b7f-a456-42a70e164474-kube-api-access-9lw2d\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:30:25.436554 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.436548 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7734961d-c56c-4b7f-a456-42a70e164474-extensions-socket-volume\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:30:25.611043 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.610950 2580 generic.go:356] "Generic (PLEG): container finished" podID="7734961d-c56c-4b7f-a456-42a70e164474" containerID="3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36" exitCode=0 Jun 06 22:30:25.611043 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.610989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" event={"ID":"7734961d-c56c-4b7f-a456-42a70e164474","Type":"ContainerDied","Data":"3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36"} Jun 06 22:30:25.611043 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.611006 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" Jun 06 22:30:25.611043 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.611025 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9" event={"ID":"7734961d-c56c-4b7f-a456-42a70e164474","Type":"ContainerDied","Data":"7ba00663440f8df13b819a8ffbfe9d8ba4800399022371dadff68a02c0619962"} Jun 06 22:30:25.611043 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.611046 2580 scope.go:117] "RemoveContainer" containerID="3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36" Jun 06 22:30:25.620606 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.620589 2580 scope.go:117] "RemoveContainer" containerID="3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36" Jun 06 22:30:25.620881 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:30:25.620856 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36\": container with ID starting with 3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36 not found: ID does not exist" containerID="3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36" Jun 06 22:30:25.620958 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.620893 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36"} err="failed to get container status \"3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36\": rpc error: code = NotFound desc = could not find container \"3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36\": container with ID starting with 3fd31f7a9d4234eb6d2568e090767d3878870273bf3a70ca2b26cd284e48cf36 not found: ID does not exist" Jun 06 22:30:25.628171 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.628150 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9"] Jun 06 22:30:25.631507 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:25.631485 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mk7j9"] Jun 06 22:30:27.490421 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:27.490378 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7734961d-c56c-4b7f-a456-42a70e164474" path="/var/lib/kubelet/pods/7734961d-c56c-4b7f-a456-42a70e164474/volumes" Jun 06 22:30:49.200752 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:30:49.200714 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:31:20.376799 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:31:20.376704 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:32:53.607842 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:32:53.607814 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:32:53.614091 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:32:53.614063 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:32:53.614512 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:32:53.614490 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:32:53.620457 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:32:53.620436 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:35:49.479942 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:35:49.479904 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:37:40.323118 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:37:40.323042 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:37:40.410788 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:37:40.410758 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:37:53.641902 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:37:53.641874 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:37:53.648783 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:37:53.648761 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:37:53.649753 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:37:53.649732 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:37:53.656243 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:37:53.656225 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:40:10.819633 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:10.819597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:40:16.158001 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:16.157910 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:40:40.421942 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:40.421896 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:40:53.454147 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:53.454112 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:40:55.001166 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.001122 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk"] Jun 06 22:40:55.001761 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.001737 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7734961d-c56c-4b7f-a456-42a70e164474" containerName="manager" Jun 06 22:40:55.001873 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.001762 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7734961d-c56c-4b7f-a456-42a70e164474" containerName="manager" Jun 06 22:40:55.001933 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.001887 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7734961d-c56c-4b7f-a456-42a70e164474" containerName="manager" Jun 06 22:40:55.005296 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.005260 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.007866 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.007841 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a347eaad-openshift-default-dockercfg-djhgr\"" Jun 06 22:40:55.026579 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.026549 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk"] Jun 06 22:40:55.038006 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.037976 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:40:55.085166 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.085139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-podinfo\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.085310 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.085185 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-token\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.085310 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.085220 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-credential-socket\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.085310 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.085281 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-data\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.085442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.085313 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngbkw\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-kube-api-access-ngbkw\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.085442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.085365 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-certs\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.085442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.085394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d118b11-839a-41c1-9f98-6b155729c0c9-istiod-ca-cert\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.085442 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.085425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-socket\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.085625 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.085500 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-envoy\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.135421 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.135384 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:40:55.186401 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-data\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.186555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngbkw\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-kube-api-access-ngbkw\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.186555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-certs\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.186555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d118b11-839a-41c1-9f98-6b155729c0c9-istiod-ca-cert\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.186555 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-socket\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.186719 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186638 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-envoy\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.186775 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186722 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-podinfo\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.186855 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-token\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.186855 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186791 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-data\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.186974 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.186888 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-certs\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.187072 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.187050 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-socket\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.187158 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.187079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-credential-socket\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.187212 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.187174 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d118b11-839a-41c1-9f98-6b155729c0c9-istiod-ca-cert\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.187402 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.187375 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-credential-socket\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.189797 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.189764 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-envoy\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.189797 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.189767 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-podinfo\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.196007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.195986 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-token\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.197413 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.197392 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngbkw\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-kube-api-access-ngbkw\") pod \"e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.235422 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.235391 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:40:55.317059 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.316984 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:55.337583 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.337554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:40:55.457253 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.457229 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk"] Jun 06 22:40:55.459150 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:40:55.459117 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d118b11_839a_41c1_9f98_6b155729c0c9.slice/crio-baaac5b09fbe441c6fa967424215d1918cfd526129670a5a2f9af84eaabdf8d0 WatchSource:0}: Error finding container baaac5b09fbe441c6fa967424215d1918cfd526129670a5a2f9af84eaabdf8d0: Status 404 returned error can't find the container with id baaac5b09fbe441c6fa967424215d1918cfd526129670a5a2f9af84eaabdf8d0 Jun 06 22:40:55.461355 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.461323 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:40:55.461458 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.461388 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:40:55.461458 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.461417 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:40:55.869531 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.869493 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" event={"ID":"3d118b11-839a-41c1-9f98-6b155729c0c9","Type":"ContainerStarted","Data":"85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357"} Jun 06 22:40:55.869681 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.869561 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" event={"ID":"3d118b11-839a-41c1-9f98-6b155729c0c9","Type":"ContainerStarted","Data":"baaac5b09fbe441c6fa967424215d1918cfd526129670a5a2f9af84eaabdf8d0"} Jun 06 22:40:55.892244 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:55.892194 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" podStartSLOduration=1.892178184 podStartE2EDuration="1.892178184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:40:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:40:55.890605613 +0000 UTC m=+2282.889585110" watchObservedRunningTime="2026-06-06 22:40:55.892178184 +0000 UTC m=+2282.891157667" Jun 06 22:40:56.317488 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:56.317453 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:56.322474 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:56.322446 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:56.874634 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:56.874604 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:56.875727 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:56.875711 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:40:58.605194 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:40:58.605163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:01.200823 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:01.200769 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:01.367533 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:01.367495 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk"] Jun 06 22:41:01.367744 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:01.367707 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" podUID="3d118b11-839a-41c1-9f98-6b155729c0c9" containerName="istio-proxy" containerID="cri-o://85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357" gracePeriod=30 Jun 06 22:41:01.390525 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:01.390494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:01.489388 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:01.489268 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:03.232610 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:03.232576 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:06.629728 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.629673 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:41:06.703743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.703713 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-credential-socket\") pod \"3d118b11-839a-41c1-9f98-6b155729c0c9\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " Jun 06 22:41:06.703743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.703743 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-certs\") pod \"3d118b11-839a-41c1-9f98-6b155729c0c9\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " Jun 06 22:41:06.703952 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.703787 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-socket\") pod \"3d118b11-839a-41c1-9f98-6b155729c0c9\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " Jun 06 22:41:06.703952 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.703823 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d118b11-839a-41c1-9f98-6b155729c0c9-istiod-ca-cert\") pod \"3d118b11-839a-41c1-9f98-6b155729c0c9\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " Jun 06 22:41:06.703952 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.703845 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-token\") pod \"3d118b11-839a-41c1-9f98-6b155729c0c9\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " Jun 06 22:41:06.704066 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704017 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-data\") pod \"3d118b11-839a-41c1-9f98-6b155729c0c9\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " Jun 06 22:41:06.704066 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704017 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d118b11-839a-41c1-9f98-6b155729c0c9" (UID: "3d118b11-839a-41c1-9f98-6b155729c0c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:06.704140 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704041 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d118b11-839a-41c1-9f98-6b155729c0c9" (UID: "3d118b11-839a-41c1-9f98-6b155729c0c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:06.704140 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704055 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-envoy\") pod \"3d118b11-839a-41c1-9f98-6b155729c0c9\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " Jun 06 22:41:06.704140 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704122 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngbkw\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-kube-api-access-ngbkw\") pod \"3d118b11-839a-41c1-9f98-6b155729c0c9\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " Jun 06 22:41:06.704262 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704137 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d118b11-839a-41c1-9f98-6b155729c0c9" (UID: "3d118b11-839a-41c1-9f98-6b155729c0c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:06.704262 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704173 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-podinfo\") pod \"3d118b11-839a-41c1-9f98-6b155729c0c9\" (UID: \"3d118b11-839a-41c1-9f98-6b155729c0c9\") " Jun 06 22:41:06.704262 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704238 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d118b11-839a-41c1-9f98-6b155729c0c9" (UID: "3d118b11-839a-41c1-9f98-6b155729c0c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:06.704262 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704247 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d118b11-839a-41c1-9f98-6b155729c0c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d118b11-839a-41c1-9f98-6b155729c0c9" (UID: "3d118b11-839a-41c1-9f98-6b155729c0c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:41:06.704514 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704496 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-data\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:06.704579 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704515 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-credential-socket\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:06.704579 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704530 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-certs\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:06.704579 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704546 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-workload-socket\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:06.704579 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.704561 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d118b11-839a-41c1-9f98-6b155729c0c9-istiod-ca-cert\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:06.706221 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.706199 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d118b11-839a-41c1-9f98-6b155729c0c9" (UID: "3d118b11-839a-41c1-9f98-6b155729c0c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:41:06.706353 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.706326 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-kube-api-access-ngbkw" (OuterVolumeSpecName: "kube-api-access-ngbkw") pod "3d118b11-839a-41c1-9f98-6b155729c0c9" (UID: "3d118b11-839a-41c1-9f98-6b155729c0c9"). InnerVolumeSpecName "kube-api-access-ngbkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:41:06.706521 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.706497 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d118b11-839a-41c1-9f98-6b155729c0c9" (UID: "3d118b11-839a-41c1-9f98-6b155729c0c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:06.706614 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.706518 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d118b11-839a-41c1-9f98-6b155729c0c9" (UID: "3d118b11-839a-41c1-9f98-6b155729c0c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 06 22:41:06.805149 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.805120 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-token\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:06.805149 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.805143 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-envoy\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:06.805149 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.805153 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngbkw\" (UniqueName: \"kubernetes.io/projected/3d118b11-839a-41c1-9f98-6b155729c0c9-kube-api-access-ngbkw\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:06.805380 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.805162 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d118b11-839a-41c1-9f98-6b155729c0c9-istio-podinfo\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:06.926617 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.926586 2580 generic.go:356] "Generic (PLEG): container finished" podID="3d118b11-839a-41c1-9f98-6b155729c0c9" containerID="85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357" exitCode=0 Jun 06 22:41:06.926784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.926634 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" Jun 06 22:41:06.926784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.926675 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" event={"ID":"3d118b11-839a-41c1-9f98-6b155729c0c9","Type":"ContainerDied","Data":"85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357"} Jun 06 22:41:06.926784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.926740 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk" event={"ID":"3d118b11-839a-41c1-9f98-6b155729c0c9","Type":"ContainerDied","Data":"baaac5b09fbe441c6fa967424215d1918cfd526129670a5a2f9af84eaabdf8d0"} Jun 06 22:41:06.926784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.926762 2580 scope.go:117] "RemoveContainer" containerID="85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357" Jun 06 22:41:06.936599 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.936582 2580 scope.go:117] "RemoveContainer" containerID="85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357" Jun 06 22:41:06.936883 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:41:06.936863 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357\": container with ID starting with 85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357 not found: ID does not exist" containerID="85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357" Jun 06 22:41:06.936950 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.936890 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357"} err="failed to get container status \"85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357\": rpc error: code = NotFound desc = could not find container \"85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357\": container with ID starting with 85e46e640ecad996969bd2303b6eae23e3d9e9c8527b1ec927c216bdbcdb5357 not found: ID does not exist" Jun 06 22:41:06.948831 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.948793 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk"] Jun 06 22:41:06.952816 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:06.952783 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a347eaad-openshift-default-6f5747f778-w9fpk"] Jun 06 22:41:07.234326 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.234238 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:07.408365 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.408328 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:07.431541 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.431503 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf"] Jun 06 22:41:07.432251 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.432223 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d118b11-839a-41c1-9f98-6b155729c0c9" containerName="istio-proxy" Jun 06 22:41:07.432251 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.432253 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d118b11-839a-41c1-9f98-6b155729c0c9" containerName="istio-proxy" Jun 06 22:41:07.432471 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.432374 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d118b11-839a-41c1-9f98-6b155729c0c9" containerName="istio-proxy" Jun 06 22:41:07.437955 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.437932 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.440558 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.440537 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4fa20bbb-openshift-default-dockercfg-mq9sh\"" Jun 06 22:41:07.448195 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.448172 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf"] Jun 06 22:41:07.481743 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.481683 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:07.489857 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.489786 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d118b11-839a-41c1-9f98-6b155729c0c9" path="/var/lib/kubelet/pods/3d118b11-839a-41c1-9f98-6b155729c0c9/volumes" Jun 06 22:41:07.511611 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.511580 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-envoy\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.511775 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.511632 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-socket\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.511775 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.511659 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-certs\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.511775 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.511724 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istiod-ca-cert\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.511775 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.511764 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-credential-socket\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.512002 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.511863 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-data\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.512002 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.511916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tz86\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-kube-api-access-7tz86\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.512002 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.511980 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-podinfo\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.512153 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.512017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-token\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.581976 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.581939 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:07.613428 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.613398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-token\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.613562 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.613441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-envoy\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.613562 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.613477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-socket\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.613562 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.613492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-certs\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.613562 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.613537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istiod-ca-cert\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.613777 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.613596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-credential-socket\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.613777 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.613658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-data\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.613777 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.613718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tz86\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-kube-api-access-7tz86\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.613777 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.613759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-podinfo\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.614081 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.614053 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-certs\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.614205 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.614163 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-credential-socket\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.614291 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.614269 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-data\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.614348 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.614311 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istiod-ca-cert\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.614408 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.614342 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-socket\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.615844 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.615827 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-envoy\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.616274 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.616258 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-podinfo\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.625448 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.625423 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-token\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.625785 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.625762 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tz86\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-kube-api-access-7tz86\") pod \"e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.680608 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.680580 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:07.751049 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.750961 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:07.894570 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.894541 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf"] Jun 06 22:41:07.896583 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:41:07.896556 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d4df6aa_1da3_465a_9934_5d293e0c16dc.slice/crio-322fe630027f7f6abc8e3eb7605e92475fbc32c50a0b78ef5cdda1454a326517 WatchSource:0}: Error finding container 322fe630027f7f6abc8e3eb7605e92475fbc32c50a0b78ef5cdda1454a326517: Status 404 returned error can't find the container with id 322fe630027f7f6abc8e3eb7605e92475fbc32c50a0b78ef5cdda1454a326517 Jun 06 22:41:07.898876 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.898844 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:41:07.898976 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.898904 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:41:07.898976 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.898948 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 06 22:41:07.933692 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:07.933662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" event={"ID":"9d4df6aa-1da3-465a-9934-5d293e0c16dc","Type":"ContainerStarted","Data":"322fe630027f7f6abc8e3eb7605e92475fbc32c50a0b78ef5cdda1454a326517"} Jun 06 22:41:08.940562 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:08.940524 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" event={"ID":"9d4df6aa-1da3-465a-9934-5d293e0c16dc","Type":"ContainerStarted","Data":"ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589"} Jun 06 22:41:08.961007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:08.960963 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" podStartSLOduration=1.960946988 podStartE2EDuration="1.960946988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-06 22:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:41:08.959139932 +0000 UTC m=+2295.958119416" watchObservedRunningTime="2026-06-06 22:41:08.960946988 +0000 UTC m=+2295.959926472" Jun 06 22:41:09.751379 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:09.751338 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:09.756444 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:09.756417 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:09.945548 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:09.945523 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:09.946437 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:09.946416 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:10.676704 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:10.676674 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:13.616268 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:13.616211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:14.719970 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:14.719932 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf"] Jun 06 22:41:14.720374 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:14.720136 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" podUID="9d4df6aa-1da3-465a-9934-5d293e0c16dc" containerName="istio-proxy" containerID="cri-o://ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589" gracePeriod=30 Jun 06 22:41:14.745257 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:14.745224 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:14.823750 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:14.823715 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:16.254384 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:16.254280 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:19.977282 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:19.977256 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:20.004269 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.004183 2580 generic.go:356] "Generic (PLEG): container finished" podID="9d4df6aa-1da3-465a-9934-5d293e0c16dc" containerID="ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589" exitCode=0 Jun 06 22:41:20.004269 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.004238 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" Jun 06 22:41:20.004457 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.004270 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" event={"ID":"9d4df6aa-1da3-465a-9934-5d293e0c16dc","Type":"ContainerDied","Data":"ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589"} Jun 06 22:41:20.004457 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.004339 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf" event={"ID":"9d4df6aa-1da3-465a-9934-5d293e0c16dc","Type":"ContainerDied","Data":"322fe630027f7f6abc8e3eb7605e92475fbc32c50a0b78ef5cdda1454a326517"} Jun 06 22:41:20.004457 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.004373 2580 scope.go:117] "RemoveContainer" containerID="ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589" Jun 06 22:41:20.016264 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.016245 2580 scope.go:117] "RemoveContainer" containerID="ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589" Jun 06 22:41:20.016821 ip-10-0-139-115 kubenswrapper[2580]: E0606 22:41:20.016753 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589\": container with ID starting with ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589 not found: ID does not exist" containerID="ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589" Jun 06 22:41:20.016821 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.016793 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589"} err="failed to get container status \"ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589\": rpc error: code = NotFound desc = could not find container \"ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589\": container with ID starting with ebcec4390f850e6ae0d06cc6119c5766d2c675c85c0e15ef2305344797c79589 not found: ID does not exist" Jun 06 22:41:20.037643 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.037620 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-envoy\") pod \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " Jun 06 22:41:20.037758 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.037654 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-certs\") pod \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " Jun 06 22:41:20.037758 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.037709 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istiod-ca-cert\") pod \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " Jun 06 22:41:20.037758 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.037730 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tz86\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-kube-api-access-7tz86\") pod \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " Jun 06 22:41:20.037758 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.037753 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-token\") pod \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " Jun 06 22:41:20.038007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.037866 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-data\") pod \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " Jun 06 22:41:20.038007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.037927 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-credential-socket\") pod \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " Jun 06 22:41:20.038007 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.037957 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-socket\") pod \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " Jun 06 22:41:20.038176 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038009 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d4df6aa-1da3-465a-9934-5d293e0c16dc" (UID: "9d4df6aa-1da3-465a-9934-5d293e0c16dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:20.038176 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038032 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-podinfo\") pod \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\" (UID: \"9d4df6aa-1da3-465a-9934-5d293e0c16dc\") " Jun 06 22:41:20.038283 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038191 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d4df6aa-1da3-465a-9934-5d293e0c16dc" (UID: "9d4df6aa-1da3-465a-9934-5d293e0c16dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 06 22:41:20.038283 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038204 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d4df6aa-1da3-465a-9934-5d293e0c16dc" (UID: "9d4df6aa-1da3-465a-9934-5d293e0c16dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:20.038448 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038325 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d4df6aa-1da3-465a-9934-5d293e0c16dc" (UID: "9d4df6aa-1da3-465a-9934-5d293e0c16dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:20.038507 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038481 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-socket\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:20.038507 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038501 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-workload-certs\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:20.038619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038514 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istiod-ca-cert\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:20.038619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038527 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-credential-socket\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:20.038619 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.038585 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d4df6aa-1da3-465a-9934-5d293e0c16dc" (UID: "9d4df6aa-1da3-465a-9934-5d293e0c16dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:20.039975 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.039944 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d4df6aa-1da3-465a-9934-5d293e0c16dc" (UID: "9d4df6aa-1da3-465a-9934-5d293e0c16dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 06 22:41:20.040085 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.039977 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-kube-api-access-7tz86" (OuterVolumeSpecName: "kube-api-access-7tz86") pod "9d4df6aa-1da3-465a-9934-5d293e0c16dc" (UID: "9d4df6aa-1da3-465a-9934-5d293e0c16dc"). InnerVolumeSpecName "kube-api-access-7tz86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:41:20.040085 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.040014 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d4df6aa-1da3-465a-9934-5d293e0c16dc" (UID: "9d4df6aa-1da3-465a-9934-5d293e0c16dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 06 22:41:20.040184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.040091 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d4df6aa-1da3-465a-9934-5d293e0c16dc" (UID: "9d4df6aa-1da3-465a-9934-5d293e0c16dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 06 22:41:20.139099 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.139067 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-envoy\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:20.139099 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.139095 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tz86\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-kube-api-access-7tz86\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:20.139099 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.139105 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-token\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:20.139334 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.139115 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-data\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:20.139334 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.139123 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d4df6aa-1da3-465a-9934-5d293e0c16dc-istio-podinfo\") on node \"ip-10-0-139-115.ec2.internal\" DevicePath \"\"" Jun 06 22:41:20.329827 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.329455 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf"] Jun 06 22:41:20.336222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.336196 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4fa20bbb-openshift-default-59c556d748-twfkf"] Jun 06 22:41:20.460241 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.460185 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:20.519022 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:20.518995 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:21.489343 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:21.489306 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d4df6aa-1da3-465a-9934-5d293e0c16dc" path="/var/lib/kubelet/pods/9d4df6aa-1da3-465a-9934-5d293e0c16dc/volumes" Jun 06 22:41:30.900368 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:30.900337 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-64ff74d9d5-2jnzz_8d9a23a6-63f4-4caf-82b8-4528af3f260c/nginx/0.log" Jun 06 22:41:31.360925 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:31.360834 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6897b7b5db-trrtt_73c2cb19-a36f-4fdb-8595-eaab68a01b12/maas-api/0.log" Jun 06 22:41:31.574784 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:31.574747 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-844f958585-g5bsk_2f9ce1cb-eca3-480f-8cad-c1cd57590f63/manager/0.log" Jun 06 22:41:32.026298 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:32.026268 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c76bcc58-lk6p8_5e0e609d-72e2-4d57-9139-48dc0560fdcf/copy-manifests/0.log" Jun 06 22:41:32.031924 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:32.031891 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c76bcc58-lk6p8_5e0e609d-72e2-4d57-9139-48dc0560fdcf/manager/0.log" Jun 06 22:41:33.570940 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:33.570911 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5j6b9_d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e/manager/0.log" Jun 06 22:41:33.888900 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:33.888799 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mhskb_154a901b-ef4b-48d0-af4c-2458da8b0080/registry-server/0.log" Jun 06 22:41:34.557349 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:34.557321 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f979554rqk_2f4537ef-f82b-470a-93e4-d1e646796f66/istio-proxy/0.log" Jun 06 22:41:34.997192 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:34.997160 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zrt7w_aeb14052-2c78-4059-9b3d-4d0c9bec4f4e/istio-proxy/1.log" Jun 06 22:41:35.104973 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:35.104936 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-h7c4p_3ce1b548-8661-433f-8a1a-401d468e7dcd/payload-pre-processing/0.log" Jun 06 22:41:35.217647 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:35.217621 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-v7nzj_2014ea99-c8c0-4bdc-906b-53670a0efcbc/payload-processing/0.log" Jun 06 22:41:35.322799 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:35.322729 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78476cbbd5-ldsl9_17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d/router/0.log" Jun 06 22:41:35.658470 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:35.658441 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l_4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e/storage-initializer/0.log" Jun 06 22:41:35.665329 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:35.665306 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-hbt6l_4a647be8-4e2c-4ae3-baf4-dbe0ce5a9f8e/main/0.log" Jun 06 22:41:35.883397 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:35.883368 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f_d0d7b6af-c5d3-4841-aeff-74ae805ae528/storage-initializer/0.log" Jun 06 22:41:35.890615 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:35.890594 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-7wg4f_d0d7b6af-c5d3-4841-aeff-74ae805ae528/main/0.log" Jun 06 22:41:35.999627 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:35.999558 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n_40569393-55f7-4d74-836a-593f190d22b1/storage-initializer/0.log" Jun 06 22:41:36.006362 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:36.006343 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cq2r9n_40569393-55f7-4d74-836a-593f190d22b1/main/0.log" Jun 06 22:41:36.113431 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:36.113406 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-glsx8_d3c8ae5d-0217-4b11-8f1d-e202391ed03e/storage-initializer/0.log" Jun 06 22:41:36.120246 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:36.120226 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-glsx8_d3c8ae5d-0217-4b11-8f1d-e202391ed03e/main/0.log" Jun 06 22:41:40.314753 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:40.314720 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:40.379151 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:40.379118 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" limit=1024 Jun 06 22:41:43.502165 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:43.502091 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-m7gsp_db6d6e2f-7de5-4121-b6ca-d7f5aa168471/global-pull-secret-syncer/0.log" Jun 06 22:41:43.547531 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:43.547506 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bkvtn_7ff591cd-bfd1-4f0e-8bc3-4a42011a49b0/konnectivity-agent/0.log" Jun 06 22:41:43.671558 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:43.671533 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-115.ec2.internal_2f312d8097474ef9a1a11d8e634ff7d8/haproxy/0.log" Jun 06 22:41:48.873479 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:48.873452 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5j6b9_d31c3b42-d7d6-4fe1-ac60-7e3b00f1198e/manager/0.log" Jun 06 22:41:48.965091 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:48.965064 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mhskb_154a901b-ef4b-48d0-af4c-2458da8b0080/registry-server/0.log" Jun 06 22:41:51.082823 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.082721 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-dzfv6_40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764/kube-state-metrics/0.log" Jun 06 22:41:51.100524 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.100501 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-dzfv6_40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764/kube-rbac-proxy-main/0.log" Jun 06 22:41:51.122222 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.122198 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-dzfv6_40bb3b22-4cfb-4bf3-863e-ac1ffdfc3764/kube-rbac-proxy-self/0.log" Jun 06 22:41:51.156799 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.156771 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5fc4968dbb-jzvwh_6096f400-3aba-4410-bf60-7ae2b9bdb31e/metrics-server/0.log" Jun 06 22:41:51.186165 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.186127 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-8pfnc_06e6a9e9-c439-423f-98da-80a7e0f105ce/monitoring-plugin/0.log" Jun 06 22:41:51.301662 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.301632 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbrjj_2294bbdf-04ad-4961-8d58-01f80a5464f2/node-exporter/0.log" Jun 06 22:41:51.321096 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.321072 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbrjj_2294bbdf-04ad-4961-8d58-01f80a5464f2/kube-rbac-proxy/0.log" Jun 06 22:41:51.341327 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.341262 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fbrjj_2294bbdf-04ad-4961-8d58-01f80a5464f2/init-textfile/0.log" Jun 06 22:41:51.561374 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.561349 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a4e87b7d-bf13-4db9-8872-32f07f762d8d/prometheus/0.log" Jun 06 22:41:51.589268 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.589240 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a4e87b7d-bf13-4db9-8872-32f07f762d8d/config-reloader/0.log" Jun 06 22:41:51.614316 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.614248 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a4e87b7d-bf13-4db9-8872-32f07f762d8d/thanos-sidecar/0.log" Jun 06 22:41:51.636562 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.636541 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a4e87b7d-bf13-4db9-8872-32f07f762d8d/kube-rbac-proxy-web/0.log" Jun 06 22:41:51.660510 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.660488 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a4e87b7d-bf13-4db9-8872-32f07f762d8d/kube-rbac-proxy/0.log" Jun 06 22:41:51.681212 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.681194 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a4e87b7d-bf13-4db9-8872-32f07f762d8d/kube-rbac-proxy-thanos/0.log" Jun 06 22:41:51.703119 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.703096 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a4e87b7d-bf13-4db9-8872-32f07f762d8d/init-config-reloader/0.log" Jun 06 22:41:51.731843 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.731815 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-5mdqd_b815c5c4-8fee-4374-a16f-8bf93b94d346/prometheus-operator/0.log" Jun 06 22:41:51.751502 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.751466 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-5mdqd_b815c5c4-8fee-4374-a16f-8bf93b94d346/kube-rbac-proxy/0.log" Jun 06 22:41:51.788926 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.788907 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-rvvb6_819aa35f-218e-4f27-b32f-6d3d4adc9e26/prometheus-operator-admission-webhook/0.log" Jun 06 22:41:51.827424 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.827401 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c97f86fb6-jpmvc_79a9d2e2-54a3-4dca-b503-0a8110954c69/telemeter-client/0.log" Jun 06 22:41:51.855229 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.855199 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c97f86fb6-jpmvc_79a9d2e2-54a3-4dca-b503-0a8110954c69/reload/0.log" Jun 06 22:41:51.879116 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:51.879061 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6c97f86fb6-jpmvc_79a9d2e2-54a3-4dca-b503-0a8110954c69/kube-rbac-proxy/0.log" Jun 06 22:41:53.571654 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:53.571626 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-hhv9q_ff32d99a-8480-4abf-91d3-8eb414d4bf85/networking-console-plugin/0.log" Jun 06 22:41:54.214237 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:54.214211 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/2.log" Jun 06 22:41:54.219208 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:54.219189 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4ngmg_d78b6164-d3a6-4217-a627-f2e0fb3ade06/console-operator/3.log" Jun 06 22:41:55.581000 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:55.580971 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-r9d9f_e7a21203-dae9-4818-a0cb-8115e6c92d19/volume-data-source-validator/0.log" Jun 06 22:41:56.016174 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.016136 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks"] Jun 06 22:41:56.016789 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.016763 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d4df6aa-1da3-465a-9934-5d293e0c16dc" containerName="istio-proxy" Jun 06 22:41:56.016789 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.016789 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d4df6aa-1da3-465a-9934-5d293e0c16dc" containerName="istio-proxy" Jun 06 22:41:56.016992 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.016916 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d4df6aa-1da3-465a-9934-5d293e0c16dc" containerName="istio-proxy" Jun 06 22:41:56.021499 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.021484 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.024184 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.024163 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pkwkm\"/\"openshift-service-ca.crt\"" Jun 06 22:41:56.025562 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.025537 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pkwkm\"/\"kube-root-ca.crt\"" Jun 06 22:41:56.026012 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.025735 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pkwkm\"/\"default-dockercfg-hl46c\"" Jun 06 22:41:56.034746 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.032305 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks"] Jun 06 22:41:56.077032 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.076989 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trdvv\" (UniqueName: \"kubernetes.io/projected/5ad9705e-f05c-4205-931f-eddb0d331d8e-kube-api-access-trdvv\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.077216 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.077041 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-proc\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.077216 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.077190 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-sys\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.077293 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.077226 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-podres\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.077328 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.077287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-lib-modules\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.178434 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.178396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trdvv\" (UniqueName: \"kubernetes.io/projected/5ad9705e-f05c-4205-931f-eddb0d331d8e-kube-api-access-trdvv\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.178596 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.178442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-proc\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.178596 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.178532 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-sys\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.178596 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.178560 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-podres\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.178713 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.178600 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-proc\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.178713 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.178605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-lib-modules\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.178713 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.178669 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-sys\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.179025 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.178715 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-lib-modules\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.179025 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.178727 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ad9705e-f05c-4205-931f-eddb0d331d8e-podres\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.185896 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.185868 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trdvv\" (UniqueName: \"kubernetes.io/projected/5ad9705e-f05c-4205-931f-eddb0d331d8e-kube-api-access-trdvv\") pod \"perf-node-gather-daemonset-rsmks\" (UID: \"5ad9705e-f05c-4205-931f-eddb0d331d8e\") " pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.336530 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.336441 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:56.471238 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.471208 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks"] Jun 06 22:41:56.473170 ip-10-0-139-115 kubenswrapper[2580]: W0606 22:41:56.473143 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ad9705e_f05c_4205_931f_eddb0d331d8e.slice/crio-7666313cad0f80d63430ca74a3acd1195cf62e5cca2045a0af2ae1ee3cceb956 WatchSource:0}: Error finding container 7666313cad0f80d63430ca74a3acd1195cf62e5cca2045a0af2ae1ee3cceb956: Status 404 returned error can't find the container with id 7666313cad0f80d63430ca74a3acd1195cf62e5cca2045a0af2ae1ee3cceb956 Jun 06 22:41:56.834675 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.834645 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ld6pc_89a055b7-3e1a-4f23-85cf-e60006d08741/dns/0.log" Jun 06 22:41:56.854857 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.854764 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ld6pc_89a055b7-3e1a-4f23-85cf-e60006d08741/kube-rbac-proxy/0.log" Jun 06 22:41:56.947664 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:56.947636 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bp8js_c6c24d13-14a9-4f94-8112-0ff9d1f62946/dns-node-resolver/0.log" Jun 06 22:41:57.198461 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:57.198426 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" event={"ID":"5ad9705e-f05c-4205-931f-eddb0d331d8e","Type":"ContainerStarted","Data":"cd2fcf8e206d546365f556dd180cd09c35d3faded1951d475132cfe0f2873741"} Jun 06 22:41:57.198611 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:57.198478 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" event={"ID":"5ad9705e-f05c-4205-931f-eddb0d331d8e","Type":"ContainerStarted","Data":"7666313cad0f80d63430ca74a3acd1195cf62e5cca2045a0af2ae1ee3cceb956"} Jun 06 22:41:57.198611 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:57.198509 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:41:57.213778 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:57.213734 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" podStartSLOduration=1.213720212 podStartE2EDuration="1.213720212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-06 22:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-06 22:41:57.212477103 +0000 UTC m=+2344.211456588" watchObservedRunningTime="2026-06-06 22:41:57.213720212 +0000 UTC m=+2344.212699695" Jun 06 22:41:57.565681 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:57.565557 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-txkmb_9862602e-db24-4b4a-ab24-a6b16129404e/node-ca/0.log" Jun 06 22:41:58.535574 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:58.535542 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f979554rqk_2f4537ef-f82b-470a-93e4-d1e646796f66/istio-proxy/0.log" Jun 06 22:41:58.728796 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:58.728764 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zrt7w_aeb14052-2c78-4059-9b3d-4d0c9bec4f4e/istio-proxy/0.log" Jun 06 22:41:58.761567 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:58.761542 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zrt7w_aeb14052-2c78-4059-9b3d-4d0c9bec4f4e/istio-proxy/1.log" Jun 06 22:41:58.796381 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:58.796309 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-h7c4p_3ce1b548-8661-433f-8a1a-401d468e7dcd/payload-pre-processing/0.log" Jun 06 22:41:58.855787 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:58.855764 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-v7nzj_2014ea99-c8c0-4bdc-906b-53670a0efcbc/payload-processing/0.log" Jun 06 22:41:58.878596 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:58.878575 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78476cbbd5-ldsl9_17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d/router/0.log" Jun 06 22:41:59.515545 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:41:59.515518 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n5r5h_51877535-3dba-451c-912e-38b3b31cab69/serve-healthcheck-canary/0.log" Jun 06 22:42:00.205726 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:00.205699 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fzq8w_56f37b5d-b920-4c7c-af0e-d35708ff9d95/kube-rbac-proxy/0.log" Jun 06 22:42:00.242420 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:00.242397 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fzq8w_56f37b5d-b920-4c7c-af0e-d35708ff9d95/exporter/0.log" Jun 06 22:42:00.280821 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:00.280784 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fzq8w_56f37b5d-b920-4c7c-af0e-d35708ff9d95/extractor/0.log" Jun 06 22:42:02.672090 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:02.672059 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-64ff74d9d5-2jnzz_8d9a23a6-63f4-4caf-82b8-4528af3f260c/nginx/0.log" Jun 06 22:42:02.883633 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:02.883600 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6897b7b5db-trrtt_73c2cb19-a36f-4fdb-8595-eaab68a01b12/maas-api/0.log" Jun 06 22:42:02.977198 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:02.977120 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-844f958585-g5bsk_2f9ce1cb-eca3-480f-8cad-c1cd57590f63/manager/0.log" Jun 06 22:42:03.214336 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:03.214302 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pkwkm/perf-node-gather-daemonset-rsmks" Jun 06 22:42:03.256524 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:03.256452 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c76bcc58-lk6p8_5e0e609d-72e2-4d57-9139-48dc0560fdcf/manager/0.log" Jun 06 22:42:03.276523 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:03.276497 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c76bcc58-lk6p8_5e0e609d-72e2-4d57-9139-48dc0560fdcf/copy-manifests/0.log" Jun 06 22:42:04.904722 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:04.904698 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7774b9f8d5-4g7zv_ee3bce4f-38ba-4bdb-855d-a563408762ff/manager/0.log" Jun 06 22:42:04.965574 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:04.965544 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-f8rcv_a0553f51-0f78-416a-911d-cc086d650427/openshift-lws-operator/0.log" Jun 06 22:42:09.289446 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:09.289415 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f979554rqk_2f4537ef-f82b-470a-93e4-d1e646796f66/istio-proxy/0.log" Jun 06 22:42:09.439321 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:09.439287 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zrt7w_aeb14052-2c78-4059-9b3d-4d0c9bec4f4e/istio-proxy/0.log" Jun 06 22:42:09.472644 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:09.472615 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zrt7w_aeb14052-2c78-4059-9b3d-4d0c9bec4f4e/istio-proxy/1.log" Jun 06 22:42:09.495098 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:09.495061 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-h7c4p_3ce1b548-8661-433f-8a1a-401d468e7dcd/payload-pre-processing/0.log" Jun 06 22:42:09.543635 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:09.543563 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-v7nzj_2014ea99-c8c0-4bdc-906b-53670a0efcbc/payload-processing/0.log" Jun 06 22:42:09.557351 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:09.557331 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78476cbbd5-ldsl9_17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d/router/0.log" Jun 06 22:42:10.710202 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:10.710175 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f979554rqk_2f4537ef-f82b-470a-93e4-d1e646796f66/istio-proxy/0.log" Jun 06 22:42:10.730139 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:10.730110 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-vsd4v_0444ca20-8bd0-4be6-a4ff-a4afef1f020e/kube-storage-version-migrator-operator/1.log" Jun 06 22:42:10.731046 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:10.731024 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-vsd4v_0444ca20-8bd0-4be6-a4ff-a4afef1f020e/kube-storage-version-migrator-operator/0.log" Jun 06 22:42:10.862105 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:10.862076 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zrt7w_aeb14052-2c78-4059-9b3d-4d0c9bec4f4e/istio-proxy/0.log" Jun 06 22:42:10.894610 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:10.894580 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zrt7w_aeb14052-2c78-4059-9b3d-4d0c9bec4f4e/istio-proxy/1.log" Jun 06 22:42:10.912079 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:10.912053 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-h7c4p_3ce1b548-8661-433f-8a1a-401d468e7dcd/payload-pre-processing/0.log" Jun 06 22:42:10.962413 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:10.962331 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-v7nzj_2014ea99-c8c0-4bdc-906b-53670a0efcbc/payload-processing/0.log" Jun 06 22:42:10.980813 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:10.980785 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78476cbbd5-ldsl9_17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d/router/0.log" Jun 06 22:42:11.949331 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:11.949303 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwmq5_22adddad-6c0c-4608-9c53-45b0644d677a/kube-multus-additional-cni-plugins/0.log" Jun 06 22:42:11.974459 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:11.974431 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwmq5_22adddad-6c0c-4608-9c53-45b0644d677a/egress-router-binary-copy/0.log" Jun 06 22:42:11.994785 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:11.994758 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwmq5_22adddad-6c0c-4608-9c53-45b0644d677a/cni-plugins/0.log" Jun 06 22:42:12.015664 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.015640 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwmq5_22adddad-6c0c-4608-9c53-45b0644d677a/bond-cni-plugin/0.log" Jun 06 22:42:12.037287 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.037261 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwmq5_22adddad-6c0c-4608-9c53-45b0644d677a/routeoverride-cni/0.log" Jun 06 22:42:12.057843 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.057791 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwmq5_22adddad-6c0c-4608-9c53-45b0644d677a/whereabouts-cni-bincopy/0.log" Jun 06 22:42:12.080517 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.080493 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwmq5_22adddad-6c0c-4608-9c53-45b0644d677a/whereabouts-cni/0.log" Jun 06 22:42:12.458649 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.458621 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-srh2f_a0e3e451-1eca-4d88-97b8-7aa14025cd9a/kube-multus/0.log" Jun 06 22:42:12.547628 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.547597 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f979554rqk_2f4537ef-f82b-470a-93e4-d1e646796f66/istio-proxy/0.log" Jun 06 22:42:12.597973 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.597938 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j7ksf_5f4eec8c-1779-4a2f-b0e0-a028da53560b/network-metrics-daemon/0.log" Jun 06 22:42:12.617457 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.617427 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j7ksf_5f4eec8c-1779-4a2f-b0e0-a028da53560b/kube-rbac-proxy/0.log" Jun 06 22:42:12.693677 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.693644 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zrt7w_aeb14052-2c78-4059-9b3d-4d0c9bec4f4e/istio-proxy/0.log" Jun 06 22:42:12.725477 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.725403 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-zrt7w_aeb14052-2c78-4059-9b3d-4d0c9bec4f4e/istio-proxy/1.log" Jun 06 22:42:12.745517 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.745489 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-h7c4p_3ce1b548-8661-433f-8a1a-401d468e7dcd/payload-pre-processing/0.log" Jun 06 22:42:12.794850 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.794789 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-v7nzj_2014ea99-c8c0-4bdc-906b-53670a0efcbc/payload-processing/0.log" Jun 06 22:42:12.808579 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:12.808556 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78476cbbd5-ldsl9_17a9ecc0-7e21-4e62-bf53-e1b3e4320d5d/router/0.log" Jun 06 22:42:13.644557 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:13.644526 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-controller/0.log" Jun 06 22:42:13.663106 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:13.663079 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/0.log" Jun 06 22:42:13.663296 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:13.663136 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/ovn-acl-logging/1.log" Jun 06 22:42:13.684668 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:13.684641 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/kube-rbac-proxy-node/0.log" Jun 06 22:42:13.707278 ip-10-0-139-115 kubenswrapper[2580]: I0606 22:42:13.707253 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74znx_c0ca3dc0-5c16-4fa6-ae87-cf4e0fd060fa/kube-rbac-proxy-ovn-metrics/0.log"